-- Logs begin at Fri 2022-06-17 12:03:57 UTC, end at Fri 2022-06-17 13:37:25 UTC. -- Jun 17 12:37:12.840515 dv-cinder-846172-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 17 12:37:12.849646 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 17 12:37:12 2022] *** Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: nodename: dv-cinder-846172-2-iscsi Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: machine: x86_64 Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: clock source: unix Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: pcre jit disabled Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: detected number of CPU cores: 4 Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: current working directory: / Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: detected binary path: /usr/bin/uwsgi-core Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: your processes number limit is 47680 Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: your memory page size is 4096 bytes Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: detected max file descriptor number: 2048 Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: lock engine: pthread robust mutexes Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: thunder lock: enabled Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 17 12:37:12.852333 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 17 12:37:12.895598 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: Python main interpreter initialized at 0x55dbfc99a9c0 Jun 17 12:37:12.895598 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: python threads support enabled Jun 17 12:37:12.895598 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: your server socket listen backlog is limited to 100 connections Jun 17 12:37:12.895598 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: your mercy for graceful operations on workers is 90 seconds Jun 17 12:37:12.896122 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: mapped 403077 bytes (393 KB) for 2 cores Jun 17 12:37:12.896122 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: *** Operational MODE: preforking *** Jun 17 12:37:12.896385 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: *** uWSGI is running in multiple interpreter mode *** Jun 17 12:37:12.896385 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: spawned uWSGI master process (pid: 96998) Jun 17 12:37:12.896545 dv-cinder-846172-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 17 12:37:12.897539 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: spawned uWSGI worker 1 (pid: 97009, cores: 1) Jun 17 12:37:12.897539 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: spawned uWSGI worker 2 (pid: 97010, cores: 1) Jun 17 12:37:12.897705 dv-cinder-846172-2-iscsi devstack@keystone.service[96998]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 17 12:37:18.359268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 12:37:18.359268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: from cryptography.utils import int_from_bytes Jun 17 12:37:18.360116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 12:37:18.360116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: from cryptography.utils import int_from_bytes Jun 17 12:37:18.360510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 12:37:18.360510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: from cryptography.utils import int_from_bytes Jun 17 12:37:18.360620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 12:37:18.360620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: from cryptography.utils import int_from_bytes Jun 17 12:37:18.409797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 12:37:18.409958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 12:37:18.409958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 12:37:18.410104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 12:37:18.410164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] command line args: [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 12:37:18.410230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] command line args: [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 12:37:18.410486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 12:37:18.410569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 12:37:18.410629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ================================================================================ {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 12:37:18.410692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ================================================================================ {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 12:37:18.410755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] admin_token = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.411181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] admin_token = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.411443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] config_dir = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.411539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] config_dir = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.411607 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.411751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.411835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] config_source = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.411972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] config_source = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.412049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] debug = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.412108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] debug = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.412164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.414662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.414757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.414820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.414883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] insecure_debug = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.414946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] insecure_debug = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.415012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.415071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.415132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.415189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.415245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] list_limit = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.415304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] list_limit = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.415900 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] log_config_append = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.416095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] log_config_append = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.416282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.416481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.416670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] log_dir = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.416864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] log_dir = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.417094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] log_file = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.417256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] log_file = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.417451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.417646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.417834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.418027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.418266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.418428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.418623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.418817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.419016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.419242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.419453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.419688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.419861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.420051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.420247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.420428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.420597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.420731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.420865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.421030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.421167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.421305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.421438 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.421579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.421746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.421899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] notification_format = cadf {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] notification_format = cadf {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.87/identity {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.87/identity {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] publish_errors = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.422980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] publish_errors = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.423136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.423280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.423448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.423587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.423716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.423849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.423992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.424149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.424286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.424421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.424548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] standard_threads = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.424675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] standard_threads = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.424854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] strict_password_check = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.424991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] strict_password_check = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.425119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.425246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.425372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] use_eventlog = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.425498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] use_eventlog = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.425632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] use_journal = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.425834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] use_journal = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.425966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] use_json = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.426110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] use_json = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.426241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] use_stderr = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.426368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] use_stderr = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.426510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] use_syslog = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.426639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] use_syslog = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.426772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] watch_log_file = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.426907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] watch_log_file = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 12:37:18.427034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.427169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.427298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.427479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.427618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.427752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.427892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.428941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.429075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.429202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.429344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.429472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.429607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.429752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.429880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.430920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.431047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.431254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.431412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.431554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.431683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.431816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.431944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.432991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.433124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.433254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.433390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.433518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.433652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.433787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.433915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.434970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.435116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.435252 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.435409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.435542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.435677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.435812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.435948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.436077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.436203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.436331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.436457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.436583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.436721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.436859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.437038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.437175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.437302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.437437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.437641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.437772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.437898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.438025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.438154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.438280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.438416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.438544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.438671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.440031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.440227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.440460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.440665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.440832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.441027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.441222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.441410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.441600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.441823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.441994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.442183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.442371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.442660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.442864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.443090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.443256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.443493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.443662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.443850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.444104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.444307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.444478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.444671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.444859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.445020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.445179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.445359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.445495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.445623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.445788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.445921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.446088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.446234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.446371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.446529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.446715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.446850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.446987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.447124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.447263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.447452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.447588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.447731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.447891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.448930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.449067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.449206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.449343 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.449471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.449598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.449726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.449865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.enabled = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.enabled = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.450999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.451129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.451257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.451420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.451555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.451690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.451828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.451957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.452086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.452226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.452356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.452484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.452613 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.452740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.452886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.453025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.453162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.453291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.453419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.453546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.453673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.453882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.454978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.455108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.455236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.455364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.455531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.455669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.455807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.455935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.456985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.457114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.457241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.457384 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.457514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.457642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.457814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.457948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.458076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.458213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.458342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] auth.external = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.458480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] auth.external = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.458608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] auth.mapped = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.458747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] auth.mapped = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.458927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.459093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.459244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.459397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.459533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] auth.password = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.459662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] auth.password = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.459791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] auth.token = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.459919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] auth.token = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] catalog.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] catalog.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.460980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.461178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.461331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.461465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.461600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.461739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.461868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.461997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] credential.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.462126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] credential.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.462271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] credential.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.462409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] credential.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.462547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.462676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.462805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.462950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.463080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.463208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.463349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.463524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.463665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.463796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.463924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.464995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.465131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.465261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.465389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.465517 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.465517 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.465776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.465916 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.466962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.467092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.467220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.467357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.467509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.467639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.467779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.467908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.468963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.469094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.469231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.469361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.469505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.469634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.469839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.469975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.470104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.470233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.470361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.470503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.470635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.470763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.470892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.471021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.471156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.471295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.471485 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.471617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.471747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.471875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.472922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.473053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.473209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.473433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.473595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.473807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.474031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.474231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.474366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.474496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.474625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.474754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.474892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.475993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.476123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.476251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.476388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.476525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.476654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.476783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.476911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.477962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.478098 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.478229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.478357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.478488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.478616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.478744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.478881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.password = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.479966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.password = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.480991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.481127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.481265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.481397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.481525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.481525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.481876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.481876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.482130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.482261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.482389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.482519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.482657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.482657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.482908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.483967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.484096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.484224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.484354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.484500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.484632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.484761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.484889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.485918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.486956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.487084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.487212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.487342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.487533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.487672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.487802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.487931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.488985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.489128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.489263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.489417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.489548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.489677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.489806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.489953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.490090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.490315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.490477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.490609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.490738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] policy.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.490877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] policy.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] receipt.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.491949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] receipt.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.492988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.493116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.493253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.493382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.493511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.493649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.493777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.493906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] revoke.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.494954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.495083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.495211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.495339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.495500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] revoke.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.495708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] role.cache_time = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.495841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.495970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] role.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.496099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.496228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] role.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.496356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] role.cache_time = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.496494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] role.list_limit = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.496624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] role.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.497518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.497654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] role.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.497792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.497923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] role.list_limit = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.498061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.498205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.498346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.498477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.498608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.498745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.498897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.499027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.499157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.499287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.499495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.499649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.499785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.499915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.500991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.501248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.501381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.501510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.501510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.501768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.501899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.502976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.503106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.503236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.503401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.503554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.503698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.503830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.503961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.504091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.504221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.504350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.504489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.504621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.504751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.504881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.505051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.505229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.505374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.505506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.505637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.505767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.505897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.cache_time = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.cache_time = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.provider = fernet {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.506959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.507090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.507232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.507363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.507529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.provider = fernet {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.507673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.507804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.507934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] trust.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.508984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.509126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.509264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.509395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] trust.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.509525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.509666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.509810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.509941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.510079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.510210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.510340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.510469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.510599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.510729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.510867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.511010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.511139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.connection = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.511277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.511461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.511598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.511729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.511859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.connection = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.512947 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.513085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.513215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.513345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.513474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.513604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.513742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.513882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.514012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.514142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.514271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.514689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.514837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.514991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.515121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.515251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.515434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.515570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.515713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.515844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.515975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.516105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.516233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.516363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.516363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=97010) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 12:37:18.516619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.516619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.516619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 12:37:18.516619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=97009) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 12:37:18.619317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.619724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.620048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.620548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.620779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.620999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.628852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.629616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.657542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.657926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.658361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.658594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.712376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.712552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.761774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.761958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.762068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.762313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.828748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.828950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.829172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.829322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.829419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.829590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.829715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.829799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.830133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.830447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.830764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.831095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.853818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.854064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.854280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.854372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.854508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.854730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.854872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.854969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.855146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.855351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.855703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.856020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.856332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.856657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.866551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.867227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.880051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.880458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.880646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.880786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.880906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.881152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.881290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.881384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.881575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=97010) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.881700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.882003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.882322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.882653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.882951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=97009) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 12:37:18.899880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.900103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 12:37:18.900424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.900629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 12:37:18.900898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.901053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.901150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.901409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.901537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 12:37:18.901620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.902033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.902326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 12:37:18.902694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.903071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.903447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.903832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.913188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.913481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.913757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.914034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.916262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.916668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.917041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.917425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.922462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.925836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.926118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.926403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.926683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.926807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.927353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.927353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.927595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.927990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.928280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.928551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.928824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.931089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.931553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.931946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.932330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.932710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.933072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.933442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.933820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.934194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.934570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.934949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.955946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.957626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.957907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.962590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.963401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.963875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.964160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.964658 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.965062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.970754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.971296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.971682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.971808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.971987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.976592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.976878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.977162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.977446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.977724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.978001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.978272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.978578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.980249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.980636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.985221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:18.985654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.986035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.986416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.986768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.987157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.987587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.987941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.998581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:18.998868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.003478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.007980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.008115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.008313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.008469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.012829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.013831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.016541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.016939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.017268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.021012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.021310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.021595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.021878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.022016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.022156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.022446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.025679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.029393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.029632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.029914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.030194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.030471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.030757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.041969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.042256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.042527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.042803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.050446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.050702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.050785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.050985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.051245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.052508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.052799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.057422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.057707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.057995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.058280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.058559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.059303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.061052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.061347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.065924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.066205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.066486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.066784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.067060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.072085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.072414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.072679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.072957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.073321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.073634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.073926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.080480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.080880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.081188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.081473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.081839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.082116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.082391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.091973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.095241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.098585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.100324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.100433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.102071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.102365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.102664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.103583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.106887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.108670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.110000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.110305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.110592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.110890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.111676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.114976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.115266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.115688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.115975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.118195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.119873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.123098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.123402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.123808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.124096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.125590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.125871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.126154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.133526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.133812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.134098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.134224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=97010) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.134445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.134727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.135001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.135284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.135694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.135988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.136257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.136531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.136788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.137060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.137335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.137606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.137883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=97010) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.141914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=97009) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 12:37:19.142190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.142464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.142745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.143023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.143437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.143722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.144001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.144273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.144546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.144815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.145085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.145338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.145604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=97009) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 12:37:19.176149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55dbfc99a9c0 pid: 97010 (default app) Jun 17 12:37:19.179726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835397c3-1dea-4490-ab77-1ebf5f08394d None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:19.180028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835397c3-1dea-4490-ab77-1ebf5f08394d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:19.180316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835397c3-1dea-4490-ab77-1ebf5f08394d None None] PATH_INFO: `/v3` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:19.183751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55dbfc99a9c0 pid: 97009 (default app) Jun 17 12:37:19.184542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1/1] 10.222.0.87 () {50 vars in 791 bytes} [Fri Jun 17 12:37:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:27.111898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993aa6c4-76c7-4435-886c-546fe08b3fdd None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:27.112765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993aa6c4-76c7-4435-886c-546fe08b3fdd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:27.112765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993aa6c4-76c7-4435-886c-546fe08b3fdd None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:27.117532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1/2] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:27.123293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:27.123790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:27.123882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:27.193108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: INFO dbcounter [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] Registered counter for database keystone Jun 17 12:37:27.219147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG oslo_db.sqlalchemy.engines [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=97010) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 12:37:27.230022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writer thread running {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 12:37:27.414191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG passlib.handlers.bcrypt [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=97010) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 12:37:27.414522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG passlib.handlers.bcrypt [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=97010) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 12:37:27.469513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:27.471136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:27.603456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 2/3] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 17 12:37:27.757231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993aa6c4-76c7-4435-886c-546fe08b3fdd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:27.834381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: INFO dbcounter [None req-993aa6c4-76c7-4435-886c-546fe08b3fdd None None] Registered counter for database keystone Jun 17 12:37:27.855635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG oslo_db.sqlalchemy.engines [None req-993aa6c4-76c7-4435-886c-546fe08b3fdd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=97009) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 12:37:27.867417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writer thread running {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 12:37:27.987955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993aa6c4-76c7-4435-886c-546fe08b3fdd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:27.993692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137c219-73a7-4e9a-a8a6-c7aec8de20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:27.994365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137c219-73a7-4e9a-a8a6-c7aec8de20ba admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:27.994651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137c219-73a7-4e9a-a8a6-c7aec8de20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:27.994932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137c219-73a7-4e9a-a8a6-c7aec8de20ba admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:28.004247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137c219-73a7-4e9a-a8a6-c7aec8de20ba admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:28.056180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f137c219-73a7-4e9a-a8a6-c7aec8de20ba admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 12:37:28.056459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137c219-73a7-4e9a-a8a6-c7aec8de20ba admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:28.066999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 2/4] 10.222.0.87 () {58 vars in 1324 bytes} [Fri Jun 17 12:37:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 12:37:28.074373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:28.100653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec774-ba4a-4f27-bbad-9f55f7dcf898 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:28.107155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1ffd-cee5-43d7-8c9f-c0135d5cfd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:28.108318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1ffd-cee5-43d7-8c9f-c0135d5cfd7e admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:28.108602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1ffd-cee5-43d7-8c9f-c0135d5cfd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:28.108887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1ffd-cee5-43d7-8c9f-c0135d5cfd7e admin admin] PATH_INFO: `/v3/projects/a8cba89e292b47ef9da2168f3c55b232` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:28.110562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1ffd-cee5-43d7-8c9f-c0135d5cfd7e admin admin] RBAC: Authorizing `identity:get_project(project_id=a8cba89e292b47ef9da2168f3c55b232)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:28.160358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING keystone.common.rbac_enforcer.enforcer [None req-973d1ffd-cee5-43d7-8c9f-c0135d5cfd7e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 12:37:28.160924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1ffd-cee5-43d7-8c9f-c0135d5cfd7e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:28.165851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 3/5] 10.222.0.87 () {56 vars in 1207 bytes} [Fri Jun 17 12:37:28 2022] GET /identity/v3/projects/a8cba89e292b47ef9da2168f3c55b232 => generated 344 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:30.276827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73548dfd-370d-4bdd-8e4d-fd2e61852216 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:30.277368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73548dfd-370d-4bdd-8e4d-fd2e61852216 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:30.277368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73548dfd-370d-4bdd-8e4d-fd2e61852216 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:30.281564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 3/6] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:37:30.288662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d737b83-d242-4db9-b4bc-b47f1ed542d7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:30.289085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d737b83-d242-4db9-b4bc-b47f1ed542d7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:30.289196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d737b83-d242-4db9-b4bc-b47f1ed542d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:30.310281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4d737b83-d242-4db9-b4bc-b47f1ed542d7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:30.311834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4d737b83-d242-4db9-b4bc-b47f1ed542d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:30.321568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 4/7] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 17 12:37:30.355056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73548dfd-370d-4bdd-8e4d-fd2e61852216 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:30.369005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73548dfd-370d-4bdd-8e4d-fd2e61852216 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:30.374129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51e040-b8d7-4ffb-8f63-f6f7b939575b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:30.374773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51e040-b8d7-4ffb-8f63-f6f7b939575b admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:30.375055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51e040-b8d7-4ffb-8f63-f6f7b939575b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:30.375345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51e040-b8d7-4ffb-8f63-f6f7b939575b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:30.383626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51e040-b8d7-4ffb-8f63-f6f7b939575b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:30.385425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51e040-b8d7-4ffb-8f63-f6f7b939575b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:30.394299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 4/8] 10.222.0.87 () {58 vars in 1324 bytes} [Fri Jun 17 12:37:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 12:37:30.400415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d737b83-d242-4db9-b4bc-b47f1ed542d7 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:30.414722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d737b83-d242-4db9-b4bc-b47f1ed542d7 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:30.420478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dab33f-a346-4892-ab7b-ae90e7a17a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:30.421352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dab33f-a346-4892-ab7b-ae90e7a17a32 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:30.421632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dab33f-a346-4892-ab7b-ae90e7a17a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:30.421902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dab33f-a346-4892-ab7b-ae90e7a17a32 admin admin] PATH_INFO: `/v3/users/546aa1cc97e64f3e87a1b21e2088baa8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:30.423479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dab33f-a346-4892-ab7b-ae90e7a17a32 admin admin] RBAC: Authorizing `identity:get_user(user_id=546aa1cc97e64f3e87a1b21e2088baa8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:30.425316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dab33f-a346-4892-ab7b-ae90e7a17a32 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:30.426880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 5/9] 10.222.0.87 () {56 vars in 1198 bytes} [Fri Jun 17 12:37:30 2022] GET /identity/v3/users/546aa1cc97e64f3e87a1b21e2088baa8 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:32.610697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb16b07-91d7-4a13-829b-aade045fab88 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:32.611273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb16b07-91d7-4a13-829b-aade045fab88 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:32.611273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb16b07-91d7-4a13-829b-aade045fab88 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:32.612653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 5/10] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:32.618782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c183f-21dc-4c68-8ac3-a6a27cf7cfb9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:32.619349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c183f-21dc-4c68-8ac3-a6a27cf7cfb9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:32.619470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c183f-21dc-4c68-8ac3-a6a27cf7cfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:32.638254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-245c183f-21dc-4c68-8ac3-a6a27cf7cfb9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:32.639541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-245c183f-21dc-4c68-8ac3-a6a27cf7cfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:32.692677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654655-f880-4abc-a5b3-47a974f3b850 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:32.692913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654655-f880-4abc-a5b3-47a974f3b850 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:32.693395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654655-f880-4abc-a5b3-47a974f3b850 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:32.694858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 6/11] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:32.700517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:32.700881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:32.701168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:32.729429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 6/12] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:32.741494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG passlib.handlers.bcrypt [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=97009) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 12:37:32.741937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG passlib.handlers.bcrypt [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=97009) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 12:37:32.752881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c183f-21dc-4c68-8ac3-a6a27cf7cfb9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:32.778514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:32.787519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:32.789124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:32.857525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c183f-21dc-4c68-8ac3-a6a27cf7cfb9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:32.888752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 7/13] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:32.891038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:32.891887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:32.892173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:32.892451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:32.903843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:32.910813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:32.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:32.916940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:37:32.922674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 7/14] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:37:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:32.923324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:33.005080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:33.014561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83b369-2f3c-4273-85b2-701c61fbdfa9 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:33.017485 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:33.027172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf309f0f-d6b8-4fbd-a0f0-4d53b52da886 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:33.053100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:33.054109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:33.054460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:33.054801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:33.064459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:33.065132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:33.065415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:33.065684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:33.066436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:33.066741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:33.068236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:33.080315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 8/15] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:33.086541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:33.122293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:33.123867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:33.186319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904187e-c9f7-40a9-90e1-c9edde08d8a2 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:33.220363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:33.221252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:33.221539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:33.221815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] PATH_INFO: `/v3/users/546aa1cc97e64f3e87a1b21e2088baa8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:33.222546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 8/16] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 12:37:33.223043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] RBAC: Authorizing `identity:get_user(user_id=546aa1cc97e64f3e87a1b21e2088baa8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:33.224579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:33.226200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 9/17] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:33 2022] GET /identity/v3/users/546aa1cc97e64f3e87a1b21e2088baa8 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:33.228905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:33.231241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:33.231862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:33.245663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:33.335220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d1930-970d-4ed5-a9c6-36b7b00691e7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:33.336911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc401e1-48bb-464e-97fd-a1ef6e254e10 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:33.369532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:33.370353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:33.370646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:33.370925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:33.371209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:33.372057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:33.372164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:33.372342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:33.372619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:33.373682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:33.375306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 10/18] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:33.381586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:33.394420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:33.395950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:37:33.417781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:33.418653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:33.419480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:37:33.419480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:37:33.419480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:37:33.419480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:33.419480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 9/19] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:33.425120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:33.464119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:33.482200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:33.484896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d46f-a5ea-4a13-901c-25cad929c151 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:33.518855 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c5f0f-31bc-4f78-99eb-f0e34701cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:33.519550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5f0f-31bc-4f78-99eb-f0e34701cc89 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:33.519842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5f0f-31bc-4f78-99eb-f0e34701cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:33.520120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5f0f-31bc-4f78-99eb-f0e34701cc89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:33.520982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c5f0f-31bc-4f78-99eb-f0e34701cc89 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.domain.id=default, user.id=546aa1cc97e64f3e87a1b21e2088baa8) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:33.521336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c5f0f-31bc-4f78-99eb-f0e34701cc89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:33.522797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c5f0f-31bc-4f78-99eb-f0e34701cc89 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:33.527302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952ec47-9330-4ba8-ac90-8ebbf9c44e22 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:33.529547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 11/20] 10.222.0.87 () {56 vars in 1322 bytes} [Fri Jun 17 12:37:33 2022] GET /identity/v3/role_assignments?user.id=546aa1cc97e64f3e87a1b21e2088baa8&scope.domain.id=default&role.id=4a6b9a3327544100929540645a797d97 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:33.563734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a34a59-2a9c-414e-af7b-d2e226188b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:33.564445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a34a59-2a9c-414e-af7b-d2e226188b02 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:33.564730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a34a59-2a9c-414e-af7b-d2e226188b02 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:33.565002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a34a59-2a9c-414e-af7b-d2e226188b02 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:33.565755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a34a59-2a9c-414e-af7b-d2e226188b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:33.566071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a34a59-2a9c-414e-af7b-d2e226188b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:33.567530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a34a59-2a9c-414e-af7b-d2e226188b02 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:33.587077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 10/21] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:36.074729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8693773-c3b8-4dde-b19c-ecd888662b0e None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.075253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8693773-c3b8-4dde-b19c-ecd888662b0e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.075459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8693773-c3b8-4dde-b19c-ecd888662b0e None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.077007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 12/22] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:36.082002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dce04-ce7d-4494-9e6b-9469a3b3818f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.082416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dce04-ce7d-4494-9e6b-9469a3b3818f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.082556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dce04-ce7d-4494-9e6b-9469a3b3818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.105987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d9c1-4de4-444b-9b46-c371314e43ab None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.106276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d9c1-4de4-444b-9b46-c371314e43ab None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.106557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d9c1-4de4-444b-9b46-c371314e43ab None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.107973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 13/23] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:37:36.112457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-965dce04-ce7d-4494-9e6b-9469a3b3818f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:36.113696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-965dce04-ce7d-4494-9e6b-9469a3b3818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:36.121088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb287c6-d84d-4dd0-92c0-806a70457bc5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.121377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb287c6-d84d-4dd0-92c0-806a70457bc5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.121662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb287c6-d84d-4dd0-92c0-806a70457bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.155850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8cb287c6-d84d-4dd0-92c0-806a70457bc5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:36.157812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8cb287c6-d84d-4dd0-92c0-806a70457bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:36.336238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 11/24] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:36.375012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 14/25] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 17 12:37:36.380467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dce04-ce7d-4494-9e6b-9469a3b3818f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:36.456495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb287c6-d84d-4dd0-92c0-806a70457bc5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:36.484468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:36.610387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dce04-ce7d-4494-9e6b-9469a3b3818f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:36.640804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:36.696949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb287c6-d84d-4dd0-92c0-806a70457bc5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:36.697140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28342e7-85b3-4a77-a9d7-dbccf61ace86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:36.698255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28342e7-85b3-4a77-a9d7-dbccf61ace86 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.698255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28342e7-85b3-4a77-a9d7-dbccf61ace86 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.698511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28342e7-85b3-4a77-a9d7-dbccf61ace86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.730279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28342e7-85b3-4a77-a9d7-dbccf61ace86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:36.733427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28342e7-85b3-4a77-a9d7-dbccf61ace86 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-d28342e7-85b3-4a77-a9d7-dbccf61ace86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:37:36.753597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:36.754501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:36.755276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:37:36.755276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:37:36.755276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:37:36.755276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:36.757761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 12/26] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:36.764071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcc4f7-bd8f-4847-a507-26a29e1d3aad None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.773259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcc4f7-bd8f-4847-a507-26a29e1d3aad None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.773840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcc4f7-bd8f-4847-a507-26a29e1d3aad None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.781668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 13/27] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:36 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:36.782824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcc4f7-bd8f-4847-a507-26a29e1d3aad None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:36.829451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb09c0-6fca-4b43-b3ba-c06fca14a973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:36.830332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb09c0-6fca-4b43-b3ba-c06fca14a973 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.830622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb09c0-6fca-4b43-b3ba-c06fca14a973 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.830901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb09c0-6fca-4b43-b3ba-c06fca14a973 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.849795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb09c0-6fca-4b43-b3ba-c06fca14a973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-83fb09c0-6fca-4b43-b3ba-c06fca14a973 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:36.868191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:36.869047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:36.869796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:37:36.874943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 15/28] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:37:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:36.882845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfad13-39b4-4906-b1d5-1addc89ab668 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:36.883355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfad13-39b4-4906-b1d5-1addc89ab668 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:36.884112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfad13-39b4-4906-b1d5-1addc89ab668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:36.922178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-94dfad13-39b4-4906-b1d5-1addc89ab668 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:36.924770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-94dfad13-39b4-4906-b1d5-1addc89ab668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:36.985070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:36.993677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcc4f7-bd8f-4847-a507-26a29e1d3aad None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:37.011531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 16/29] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:37.013181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811c824-3fe0-4f12-ae17-22b87de10208 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.013644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811c824-3fe0-4f12-ae17-22b87de10208 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.014107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811c824-3fe0-4f12-ae17-22b87de10208 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.015577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 17/30] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:37.016591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811c824-3fe0-4f12-ae17-22b87de10208 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:37.064467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45656c45-4f64-4b87-8aa0-479b18dd3bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:37.065633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656c45-4f64-4b87-8aa0-479b18dd3bdd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.066205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656c45-4f64-4b87-8aa0-479b18dd3bdd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.066657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656c45-4f64-4b87-8aa0-479b18dd3bdd None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.067898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45656c45-4f64-4b87-8aa0-479b18dd3bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:37.069546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45656c45-4f64-4b87-8aa0-479b18dd3bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:37.071138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45656c45-4f64-4b87-8aa0-479b18dd3bdd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:37.088319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 14/31] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:37.090598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457c48e-6572-4673-8dcc-3e9f37f91da3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.091064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457c48e-6572-4673-8dcc-3e9f37f91da3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.091577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457c48e-6572-4673-8dcc-3e9f37f91da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.103638 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:37.116660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7457c48e-6572-4673-8dcc-3e9f37f91da3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:37.119232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7457c48e-6572-4673-8dcc-3e9f37f91da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:37.151412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811c824-3fe0-4f12-ae17-22b87de10208 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:37.187424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:37.188142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.188429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.188711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.189593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:37.189776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:37.191184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:37.201142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 18/32] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:37.201142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:37.206862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 15/33] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:37.208064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457c48e-6572-4673-8dcc-3e9f37f91da3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:37.309703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:37.325676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457c48e-6572-4673-8dcc-3e9f37f91da3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:37.330185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97509-2e9d-44f8-ae8f-39e9e7cb4581 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:37.356634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:37.365492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:37.366174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.366460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.366552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:37.366738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.367191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.367513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.367726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:37.367815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.368534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:37.369203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:37.369951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:37.519410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 16/34] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:37:37 2022] POST /identity/v3/projects => generated 301 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:37.519410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:37.607354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 19/35] 10.222.0.87 () {60 vars in 1131 bytes} [Fri Jun 17 12:37:37 2022] POST /identity/v3/roles => generated 225 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:37.608378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:37.656013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702984a8-f6bf-4640-b0bb-1867fe0ac75a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:37.709159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e2739-bcb0-4930-b1b6-fede9ebf4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:37.710124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e2739-bcb0-4930-b1b6-fede9ebf4009 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.710409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e2739-bcb0-4930-b1b6-fede9ebf4009 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.710692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e2739-bcb0-4930-b1b6-fede9ebf4009 None admin] PATH_INFO: `/v3/users/546aa1cc97e64f3e87a1b21e2088baa8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.713344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e2739-bcb0-4930-b1b6-fede9ebf4009 None admin] RBAC: Authorizing `identity:get_user(user_id=546aa1cc97e64f3e87a1b21e2088baa8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:37.715662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e2739-bcb0-4930-b1b6-fede9ebf4009 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:37.719177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 17/36] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:37 2022] GET /identity/v3/users/546aa1cc97e64f3e87a1b21e2088baa8 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:37.720623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e05106-436a-454c-8b70-f7161344c91e None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.720924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e05106-436a-454c-8b70-f7161344c91e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.721209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e05106-436a-454c-8b70-f7161344c91e None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.722815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 18/37] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:37.725187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270e985-dc18-4aa5-8d8f-19c3dfd9c070 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.725480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270e985-dc18-4aa5-8d8f-19c3dfd9c070 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.725761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270e985-dc18-4aa5-8d8f-19c3dfd9c070 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.729074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 19/38] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:37.730198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270e985-dc18-4aa5-8d8f-19c3dfd9c070 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:37.746109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:37.749882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce7c1c-ccd8-4eb5-984d-34e205c3f338 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:37.800345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e7bfa-267a-4649-8212-f5ba27751fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:37.801602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e7bfa-267a-4649-8212-f5ba27751fef None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.801602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e7bfa-267a-4649-8212-f5ba27751fef None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.801602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e7bfa-267a-4649-8212-f5ba27751fef None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.802415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e7bfa-267a-4649-8212-f5ba27751fef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:37.804112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e7bfa-267a-4649-8212-f5ba27751fef None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:37.821540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 20/39] 10.222.0.87 () {60 vars in 1131 bytes} [Fri Jun 17 12:37:37 2022] POST /identity/v3/roles => generated 219 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:37.821829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21bc5e-b253-49c7-b5e5-1d7f12cd0ef9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.822171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21bc5e-b253-49c7-b5e5-1d7f12cd0ef9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.822333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21bc5e-b253-49c7-b5e5-1d7f12cd0ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.842215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0c21bc5e-b253-49c7-b5e5-1d7f12cd0ef9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:37.843861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0c21bc5e-b253-49c7-b5e5-1d7f12cd0ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:37.877868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270e985-dc18-4aa5-8d8f-19c3dfd9c070 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:37.913213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:37.921242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49704d58-df45-41f1-b37e-65b0f781e326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:37.922148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704d58-df45-41f1-b37e-65b0f781e326 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.922362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704d58-df45-41f1-b37e-65b0f781e326 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.922641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704d58-df45-41f1-b37e-65b0f781e326 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.924141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49704d58-df45-41f1-b37e-65b0f781e326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:37.925744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49704d58-df45-41f1-b37e-65b0f781e326 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:37.932475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 20/40] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:37.932475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4a894-8446-402a-905e-72f81ceb4b13 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:37.932475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4a894-8446-402a-905e-72f81ceb4b13 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:37.932475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4a894-8446-402a-905e-72f81ceb4b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:37.949624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 21/41] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:37.950388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21bc5e-b253-49c7-b5e5-1d7f12cd0ef9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:38.064452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-98c4a894-8446-402a-905e-72f81ceb4b13 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:38.065978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-98c4a894-8446-402a-905e-72f81ceb4b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:38.073220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21bc5e-b253-49c7-b5e5-1d7f12cd0ef9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:38.106751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:38.108333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:38.108622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:38.108904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] PATH_INFO: `/v3/domains/default/users/546aa1cc97e64f3e87a1b21e2088baa8/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:38.114817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=546aa1cc97e64f3e87a1b21e2088baa8, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:38.114817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:38.131462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 22/42] 10.222.0.87 () {58 vars in 1361 bytes} [Fri Jun 17 12:37:37 2022] PUT /identity/v3/domains/default/users/546aa1cc97e64f3e87a1b21e2088baa8/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:37:38.132267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:38.140695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 21/43] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:38.164561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4a894-8446-402a-905e-72f81ceb4b13 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:38.173498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:38.179271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:38.255648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d645d-6c23-4380-aedc-4790ca68c582 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:38.279985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4a894-8446-402a-905e-72f81ceb4b13 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:38.294756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:38.295729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:38.296014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:38.296440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:38.297280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:38.300437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:38.316324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 23/44] 10.222.0.87 () {60 vars in 1131 bytes} [Fri Jun 17 12:37:38 2022] POST /identity/v3/roles => generated 223 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:38.322873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3b400-bfa2-4a07-928b-aa47f17ee048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:38.323756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3b400-bfa2-4a07-928b-aa47f17ee048 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:38.324980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3b400-bfa2-4a07-928b-aa47f17ee048 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:38.324980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3b400-bfa2-4a07-928b-aa47f17ee048 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:38.326764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3b400-bfa2-4a07-928b-aa47f17ee048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:38.327143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3b400-bfa2-4a07-928b-aa47f17ee048 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:38.328942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 22/45] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:38.335229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:38.455469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:38.463705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b855e2-e297-4302-98f8-2a6b26816f8d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:38.503861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c8fa5-5944-417d-bcef-c3cca1265e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:38.504808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c8fa5-5944-417d-bcef-c3cca1265e52 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:38.505235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c8fa5-5944-417d-bcef-c3cca1265e52 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:38.505520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c8fa5-5944-417d-bcef-c3cca1265e52 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:38.506295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c8fa5-5944-417d-bcef-c3cca1265e52 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:38.507805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c8fa5-5944-417d-bcef-c3cca1265e52 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:38.538312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 24/46] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:37:38 2022] POST /identity/v3/projects => generated 312 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:37:40.323005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff74f68-7c19-428e-b732-87f850d25741 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:40.323582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff74f68-7c19-428e-b732-87f850d25741 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:40.323582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff74f68-7c19-428e-b732-87f850d25741 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:40.324958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 23/47] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:40.330706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815959dd-0a93-46ba-bc54-44d9e39992c3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:40.331181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815959dd-0a93-46ba-bc54-44d9e39992c3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:40.331319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815959dd-0a93-46ba-bc54-44d9e39992c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:40.349164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-815959dd-0a93-46ba-bc54-44d9e39992c3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:40.350763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-815959dd-0a93-46ba-bc54-44d9e39992c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:40.421592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 25/48] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:40.445544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff74f68-7c19-428e-b732-87f850d25741 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:40.529353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:40.558589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff74f68-7c19-428e-b732-87f850d25741 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:40.594996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db86851-12b8-4a68-ada6-5d6b51635bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:40.595838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db86851-12b8-4a68-ada6-5d6b51635bba None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:40.596130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db86851-12b8-4a68-ada6-5d6b51635bba None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:40.596408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db86851-12b8-4a68-ada6-5d6b51635bba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:40.608539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db86851-12b8-4a68-ada6-5d6b51635bba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-7db86851-12b8-4a68-ada6-5d6b51635bba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:40.617503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:40.618419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:40.624178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 24/49] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:37:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:40.624936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815959dd-0a93-46ba-bc54-44d9e39992c3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:40.722447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:40.726211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50853f-2f38-49fa-82e9-5ed6d802a1a3 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:40.726543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50853f-2f38-49fa-82e9-5ed6d802a1a3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:40.726829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50853f-2f38-49fa-82e9-5ed6d802a1a3 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:40.728427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 25/50] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:40.733428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9387cd-91ae-4928-b5cf-063497cb6273 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:40.733724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9387cd-91ae-4928-b5cf-063497cb6273 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:40.734008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9387cd-91ae-4928-b5cf-063497cb6273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:40.749709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815959dd-0a93-46ba-bc54-44d9e39992c3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:40.752306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3d9387cd-91ae-4928-b5cf-063497cb6273 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:40.753535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3d9387cd-91ae-4928-b5cf-063497cb6273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:40.791583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:40.792280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:40.792615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:40.792942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:40.793756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:40.794092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:40.795705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:40.805862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 26/51] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:40.812129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:40.833292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 26/52] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:40.857634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9387cd-91ae-4928-b5cf-063497cb6273 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:40.918161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eafe5-0b96-4d00-bb7b-136553034210 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:40.931116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:40.950521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:40.951426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:40.951723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:40.951999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] PATH_INFO: `/v3/users/546aa1cc97e64f3e87a1b21e2088baa8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:40.953223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] RBAC: Authorizing `identity:get_user(user_id=546aa1cc97e64f3e87a1b21e2088baa8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:40.954708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:40.956378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 27/53] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:40 2022] GET /identity/v3/users/546aa1cc97e64f3e87a1b21e2088baa8 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:40.962075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:40.968945 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9387cd-91ae-4928-b5cf-063497cb6273 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:40.972597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:41.005058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:41.005865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:41.006346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:41.006658 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:41.007958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:41.009445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:41.011112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 27/54] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:41.016608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:41.061635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16e19a-8d33-4dcf-ba8f-2f566f46ad8f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:41.094945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:41.095771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:41.096057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:41.096335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:41.097561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:41.099030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:41.100920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 28/55] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:41.106755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:41.118067 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d7f3-5023-4fb2-8680-9ce98f2e0451 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:41.147568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:41.152825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79837eb-5d05-43f7-be31-eb25a3e8c08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:41.153641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79837eb-5d05-43f7-be31-eb25a3e8c08c None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:41.153943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79837eb-5d05-43f7-be31-eb25a3e8c08c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:41.154382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79837eb-5d05-43f7-be31-eb25a3e8c08c None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:41.155114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79837eb-5d05-43f7-be31-eb25a3e8c08c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:41.156644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79837eb-5d05-43f7-be31-eb25a3e8c08c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:41.182892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 28/56] 10.222.0.87 () {60 vars in 1140 bytes} [Fri Jun 17 12:37:41 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:41.184610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:41.210308 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08572-19ba-41f2-b118-a80ff329b0a3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:41.248001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da927e-06d9-48a9-a76b-b9f7d07b7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:41.248686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da927e-06d9-48a9-a76b-b9f7d07b7f45 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:41.248978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da927e-06d9-48a9-a76b-b9f7d07b7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:41.249261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da927e-06d9-48a9-a76b-b9f7d07b7f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:41.250122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da927e-06d9-48a9-a76b-b9f7d07b7f45 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.domain.id=default, user.id=546aa1cc97e64f3e87a1b21e2088baa8) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:41.250444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da927e-06d9-48a9-a76b-b9f7d07b7f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:41.252349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da927e-06d9-48a9-a76b-b9f7d07b7f45 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:41.259724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 29/57] 10.222.0.87 () {56 vars in 1322 bytes} [Fri Jun 17 12:37:41 2022] GET /identity/v3/role_assignments?user.id=546aa1cc97e64f3e87a1b21e2088baa8&scope.domain.id=default&role.id=4a6b9a3327544100929540645a797d97 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:43.284579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d947d-e40c-47ed-8961-16de36386c6f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:43.285154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d947d-e40c-47ed-8961-16de36386c6f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:43.285154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d947d-e40c-47ed-8961-16de36386c6f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:43.286530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 29/58] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:43.293507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a600cea-3d54-42af-84c0-95c394cd3825 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:43.294208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a600cea-3d54-42af-84c0-95c394cd3825 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:43.294376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a600cea-3d54-42af-84c0-95c394cd3825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:43.322092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3a600cea-3d54-42af-84c0-95c394cd3825 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:43.323320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3a600cea-3d54-42af-84c0-95c394cd3825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:43.395670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 30/59] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:43.420690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d947d-e40c-47ed-8961-16de36386c6f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:43.530553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d947d-e40c-47ed-8961-16de36386c6f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:43.565456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330e6e5-13a2-42f1-b6ec-288a990c5657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:43.566245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330e6e5-13a2-42f1-b6ec-288a990c5657 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:43.566539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330e6e5-13a2-42f1-b6ec-288a990c5657 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:43.566808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330e6e5-13a2-42f1-b6ec-288a990c5657 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:43.568132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330e6e5-13a2-42f1-b6ec-288a990c5657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:43.569742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330e6e5-13a2-42f1-b6ec-288a990c5657 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:43.571412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 30/60] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:43.577088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a600cea-3d54-42af-84c0-95c394cd3825 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:43.646283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:43.682603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a600cea-3d54-42af-84c0-95c394cd3825 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:43.721547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538d1ea-93b7-46e5-b116-074f26d2434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:43.722213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538d1ea-93b7-46e5-b116-074f26d2434e None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:43.722500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538d1ea-93b7-46e5-b116-074f26d2434e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:43.722782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538d1ea-93b7-46e5-b116-074f26d2434e None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:43.723531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538d1ea-93b7-46e5-b116-074f26d2434e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:43.725151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538d1ea-93b7-46e5-b116-074f26d2434e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:43.758016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 31/61] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:37:43 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:47.104015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f10f-2e29-4710-9776-d8fe31ba3739 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.104541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f10f-2e29-4710-9776-d8fe31ba3739 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.104615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f10f-2e29-4710-9776-d8fe31ba3739 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.106043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 31/62] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:47.121199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e10a2-783f-42e4-aa88-662c7c914735 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.121675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e10a2-783f-42e4-aa88-662c7c914735 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.121818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e10a2-783f-42e4-aa88-662c7c914735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.146294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177b55-915f-46ec-8bb6-84ddd0173a88 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.146587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177b55-915f-46ec-8bb6-84ddd0173a88 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.146901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177b55-915f-46ec-8bb6-84ddd0173a88 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.148321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 32/63] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:47.155455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3d6e10a2-783f-42e4-aa88-662c7c914735 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:47.155608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9cf63-881d-4279-aff0-231873b50b5c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.155608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9cf63-881d-4279-aff0-231873b50b5c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.155608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9cf63-881d-4279-aff0-231873b50b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.155846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3d6e10a2-783f-42e4-aa88-662c7c914735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:47.185105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-48e9cf63-881d-4279-aff0-231873b50b5c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:47.186660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-48e9cf63-881d-4279-aff0-231873b50b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:47.250910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 32/64] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:47.283798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:47.309387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 33/65] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:47.312486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e10a2-783f-42e4-aa88-662c7c914735 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:47.350758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9cf63-881d-4279-aff0-231873b50b5c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:47.372793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:47.427210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e10a2-783f-42e4-aa88-662c7c914735 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:47.468083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9cf63-881d-4279-aff0-231873b50b5c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:47.478318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5281bc9-4642-41ba-80fa-464ae9066106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:47.479159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5281bc9-4642-41ba-80fa-464ae9066106 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.479471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5281bc9-4642-41ba-80fa-464ae9066106 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.479763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5281bc9-4642-41ba-80fa-464ae9066106 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.490680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5281bc9-4642-41ba-80fa-464ae9066106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f5281bc9-4642-41ba-80fa-464ae9066106 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:47.499138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:47.500035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:37:47.500784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 33/66] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:47.501858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ade6-d898-463b-b0e7-67fc69c87723 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.502138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ade6-d898-463b-b0e7-67fc69c87723 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.502615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ade6-d898-463b-b0e7-67fc69c87723 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.504118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 34/67] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:47.508490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701d8a4-8085-4d0b-8025-7096e707b07c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.508783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701d8a4-8085-4d0b-8025-7096e707b07c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.508964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701d8a4-8085-4d0b-8025-7096e707b07c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.510236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 35/68] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:47.512093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701d8a4-8085-4d0b-8025-7096e707b07c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:47.518467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d17c-4b5b-4b56-80d4-81e578f00246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:47.519282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d17c-4b5b-4b56-80d4-81e578f00246 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.519612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d17c-4b5b-4b56-80d4-81e578f00246 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.519892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d17c-4b5b-4b56-80d4-81e578f00246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.531903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5d17c-4b5b-4b56-80d4-81e578f00246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-63b5d17c-4b5b-4b56-80d4-81e578f00246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:47.540052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:47.540883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:47.541647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 34/69] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:37:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:47.542540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e124446-433e-4baf-b554-71d2099282cb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.542981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e124446-433e-4baf-b554-71d2099282cb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.543266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e124446-433e-4baf-b554-71d2099282cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.559900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0e124446-433e-4baf-b554-71d2099282cb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:47.561097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0e124446-433e-4baf-b554-71d2099282cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:47.598074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:47.626160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701d8a4-8085-4d0b-8025-7096e707b07c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:47.636355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 35/70] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:47.638039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda6039-721d-48e3-8bb6-51f854b669a9 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.638342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda6039-721d-48e3-8bb6-51f854b669a9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.638626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda6039-721d-48e3-8bb6-51f854b669a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.653495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:47.659461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-2fda6039-721d-48e3-8bb6-51f854b669a9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:47.660960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2fda6039-721d-48e3-8bb6-51f854b669a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:47.677353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:47.678644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.679140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.679749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.681073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:47.681560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:47.683640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:47.692770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 36/71] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:47.693579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:47.730183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 36/72] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:47.732150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda6039-721d-48e3-8bb6-51f854b669a9 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:47.793174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727246-9e3c-4b29-9dfe-0958511f43b5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:47.826328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcd692-603b-4b1f-84b1-d4bb7d3af314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:47.827005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd692-603b-4b1f-84b1-d4bb7d3af314 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.827288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd692-603b-4b1f-84b1-d4bb7d3af314 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.827592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd692-603b-4b1f-84b1-d4bb7d3af314 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.828348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcd692-603b-4b1f-84b1-d4bb7d3af314 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:47.828660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcd692-603b-4b1f-84b1-d4bb7d3af314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:47.829430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda6039-721d-48e3-8bb6-51f854b669a9 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:47.830065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcd692-603b-4b1f-84b1-d4bb7d3af314 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:47.831631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:47.838703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 37/73] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:47.839878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bcceb-96b9-4afe-bea2-2b06f9ee0dff None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.840170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bcceb-96b9-4afe-bea2-2b06f9ee0dff None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.840458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bcceb-96b9-4afe-bea2-2b06f9ee0dff None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.841685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 38/74] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:47.842382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bcceb-96b9-4afe-bea2-2b06f9ee0dff None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:47.870649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:47.871510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.871797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.872076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.883315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:47.884502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:47.891389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:47.892227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:47.893153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 37/75] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:37:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:47.893968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:47.950398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bcceb-96b9-4afe-bea2-2b06f9ee0dff None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:47.983456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:47.984354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:47.984639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:47.984917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:47.995504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68d154-85bd-4f46-be98-3dbeb8b30e96 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:47.997975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:47.999503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.001166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 39/76] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:47 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.001925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.029489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e28446-1189-4c9b-ac32-1a3f62944da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.030299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28446-1189-4c9b-ac32-1a3f62944da7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.030588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28446-1189-4c9b-ac32-1a3f62944da7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.030872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28446-1189-4c9b-ac32-1a3f62944da7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.041440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e28446-1189-4c9b-ac32-1a3f62944da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-45e28446-1189-4c9b-ac32-1a3f62944da7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:48.049212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:48.050129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:48.051061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 38/77] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:48.051885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86100224-233d-4cb9-8366-4c7417609386 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.051885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86100224-233d-4cb9-8366-4c7417609386 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.052123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86100224-233d-4cb9-8366-4c7417609386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.069601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-86100224-233d-4cb9-8366-4c7417609386 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:48.070812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-86100224-233d-4cb9-8366-4c7417609386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:48.087849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.099287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.104158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bebc3-2564-4a69-b772-a7cb840f6812 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.138997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.139860 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.140142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.140416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] PATH_INFO: `/v3/users/546aa1cc97e64f3e87a1b21e2088baa8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.140964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 39/78] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:48.141813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86100224-233d-4cb9-8366-4c7417609386 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.144292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] RBAC: Authorizing `identity:get_user(user_id=546aa1cc97e64f3e87a1b21e2088baa8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.146595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.148479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 40/79] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/users/546aa1cc97e64f3e87a1b21e2088baa8 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.153531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.243517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86100224-233d-4cb9-8366-4c7417609386 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.251565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d1e0a-2182-4f92-9f6b-6f57e7af751e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.277392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.278072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.278336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.278615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.279403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:48.279716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.281119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.285674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.286329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.286619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.286899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.294337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 40/80] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.295148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.312782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.320890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.357594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.359099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.395518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74920c0d-c44d-43ba-96f2-6b8c07ae7c60 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.428973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.429647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.429940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.430224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.430981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] RBAC: Adding query filter params (name=member) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:48.431294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.432748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.440951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 41/81] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.444232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.461618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 41/82] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:37:48.462466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.521607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.527752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.544971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761c4f5-39fd-415c-84c0-7309ba8a0329 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.563334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188dc19-f5c0-46f1-b97c-ab1d29bd266d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.580661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.581321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.581602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.581878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.597692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.598509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.598801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.599083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.600543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.602030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.603907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 42/83] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.604475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.650786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.652236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.703764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6188d-249b-4933-8f27-effb519023a3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.728851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.737840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.738734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.739021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.739303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.740536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.742267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.743955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 43/84] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.744715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.750881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.759801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 42/85] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:37:48.759801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.846585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0133af6-ecfd-4711-8f0d-b1d8932a1c36 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.857787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc3591-5fc6-4ec0-b4a1-f87b9d6424eb None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:48.888525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.889417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.889719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.890001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.891252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.892867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.894244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:48.894724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 44/86] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.895204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:48.895292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.895529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:48.895878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:48.900380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:48.901912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:48.903614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 43/87] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:48.908697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:48.954624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.966164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:48.995794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8209eb-3f6d-49a1-807a-2ddc95d97a26 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:49.009261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e1e9b-3122-438a-817c-db001cce1ede None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:49.028286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:49.028968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:49.029259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:49.029539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:49.030269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:49.031769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:49.042950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:49.043660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:49.043932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:49.044225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:49.051448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 45/88] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:37:48 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:49.052259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:49.145812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:49.147477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:49.170924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894900-2bd5-42c8-bfb0-677e5f56d553 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:49.197242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:49.200351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:49.210239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:49.211153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:49.211486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:49.211770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:49.213034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:49.214534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:49.216374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 46/89] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:49 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:49.220330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:49.264512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 44/90] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:37:49.265231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:49.344359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d8fd0-8e2a-42c7-9669-1080990f15ee None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:49.380023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:49.380686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:49.380976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:49.381253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:49.396179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af9c4-3728-470c-82c9-6538698b8b0a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:49.440504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:49.441101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:49.441700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:49.441981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:49.443126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=1e936b4c92074da5a0a5d5ff77dda5ea) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:49.443886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:49.446555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:49.449349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:49.458243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:49.459908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:49.463435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 45/91] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:37:49 2022] GET /identity/v3/role_assignments?user.id=1e936b4c92074da5a0a5d5ff77dda5ea&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=fed756b9361747358b9a5948eb9a1442 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:49.464516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:49.480294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:49.642699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 47/92] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:37:49.644119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:49.703245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db58b6-7501-43e2-b1bb-93bbeb145a4a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:49.746970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:49.747708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:49.747997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:49.748280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:49.749147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=546aa1cc97e64f3e87a1b21e2088baa8) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:49.749467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:49.750938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:49.767203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:49.770921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cee3be-0bb2-4fa7-aca9-fb384885dbcd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:49.773450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 46/93] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:37:49 2022] GET /identity/v3/role_assignments?user.id=546aa1cc97e64f3e87a1b21e2088baa8&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:49.774174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:49.821728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:49.823017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:49.823543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:49.825316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:49.828202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:49.830541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:49.840672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 48/94] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:49 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:49.841791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:49.964347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:50.029905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652f4f8-f40d-4c76-8d97-82c6e0d43949 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:50.034123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43404c23-b2e3-4dbe-8da5-040ebc23af03 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:50.082799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:50.083871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:50.084384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:50.084883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:50.086630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] RBAC: Adding query filter params (role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=1e936b4c92074da5a0a5d5ff77dda5ea) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:50.089109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:50.090619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:50.099758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 49/95] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:37:49 2022] GET /identity/v3/role_assignments?user.id=1e936b4c92074da5a0a5d5ff77dda5ea&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=20865fed4c894cccb7f1c223e95353ec => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:50.107214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d22ad1-b990-48a0-9790-682313cc62f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:50.108179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22ad1-b990-48a0-9790-682313cc62f5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:50.108457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22ad1-b990-48a0-9790-682313cc62f5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:50.108733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22ad1-b990-48a0-9790-682313cc62f5 None admin] PATH_INFO: `/v3/projects/6e5427facf27427dabca7ffa81326774` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:50.110150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d22ad1-b990-48a0-9790-682313cc62f5 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5427facf27427dabca7ffa81326774)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:50.111692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d22ad1-b990-48a0-9790-682313cc62f5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:50.113526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 47/96] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:49 2022] GET /identity/v3/projects/6e5427facf27427dabca7ffa81326774 => generated 312 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:50.124070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:50.180277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:50.290377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499370ae-4152-4e40-8386-9c5ef520ddb5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:50.337556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a1dbd-620e-4bf7-8be7-b0ebc7dd4560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:50.338249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a1dbd-620e-4bf7-8be7-b0ebc7dd4560 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:50.338540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a1dbd-620e-4bf7-8be7-b0ebc7dd4560 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:50.338821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a1dbd-620e-4bf7-8be7-b0ebc7dd4560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:50.339721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a1dbd-620e-4bf7-8be7-b0ebc7dd4560 None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=6e5427facf27427dabca7ffa81326774, user.id=1e936b4c92074da5a0a5d5ff77dda5ea) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:50.340039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a1dbd-620e-4bf7-8be7-b0ebc7dd4560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:50.347835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a1dbd-620e-4bf7-8be7-b0ebc7dd4560 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:50.355023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 50/97] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:37:50 2022] GET /identity/v3/role_assignments?user.id=1e936b4c92074da5a0a5d5ff77dda5ea&scope.project.id=6e5427facf27427dabca7ffa81326774&role.id=fed756b9361747358b9a5948eb9a1442 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:51.968481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca663c8-ee34-402f-9b2f-2f49b3f06781 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:51.969101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca663c8-ee34-402f-9b2f-2f49b3f06781 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:51.969101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca663c8-ee34-402f-9b2f-2f49b3f06781 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:51.970435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 48/98] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:51.975997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29f921-5ebc-4509-be93-ef3dccf3856e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:51.976759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29f921-5ebc-4509-be93-ef3dccf3856e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:51.977188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29f921-5ebc-4509-be93-ef3dccf3856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:51.994500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-cd29f921-5ebc-4509-be93-ef3dccf3856e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:51.997138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2086208-1e10-40e8-bdfc-68583aea6a0e None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:51.997645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2086208-1e10-40e8-bdfc-68583aea6a0e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:51.998135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2086208-1e10-40e8-bdfc-68583aea6a0e None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.000079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 49/99] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:52.000989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cd29f921-5ebc-4509-be93-ef3dccf3856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:52.008569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77296dd9-974c-49d5-a737-8f3af360549b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.009129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77296dd9-974c-49d5-a737-8f3af360549b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.009608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77296dd9-974c-49d5-a737-8f3af360549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.035323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-77296dd9-974c-49d5-a737-8f3af360549b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:52.038768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-77296dd9-974c-49d5-a737-8f3af360549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:52.116525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 51/100] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:52.142335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29f921-5ebc-4509-be93-ef3dccf3856e None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:52.165354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:52.199262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 50/101] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:37:52.200444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f8b3-b4a2-4193-88cf-199566f47c9a None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.200737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f8b3-b4a2-4193-88cf-199566f47c9a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.201020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f8b3-b4a2-4193-88cf-199566f47c9a None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.202428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 51/102] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:52.209135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287019f0-f494-462f-90b0-b5256026741a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.209418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287019f0-f494-462f-90b0-b5256026741a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.209724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287019f0-f494-462f-90b0-b5256026741a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.230394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-287019f0-f494-462f-90b0-b5256026741a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:52.232266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-287019f0-f494-462f-90b0-b5256026741a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:52.276231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29f921-5ebc-4509-be93-ef3dccf3856e None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:52.308992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:52.323575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 52/103] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:37:52.323575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287019f0-f494-462f-90b0-b5256026741a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:52.333911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:52.334793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.335082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.335361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.336706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:52.338181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:52.340287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 52/104] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:52.347805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:52.456070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:52.472003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fa58a-7876-4391-a5d3-73126a2ead99 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:52.541793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:52.542852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.543346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.543844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.544780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:52.547067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:52.570777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287019f0-f494-462f-90b0-b5256026741a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:52.579470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 53/105] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:37:52 2022] POST /identity/v3/projects => generated 302 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:52.580702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:52.652667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:52.653517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.654057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.654057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.692231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:52.700480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:52.701432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:52.702211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 53/106] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:52.703599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0b0de-4643-4a14-bf69-d665b8ef897d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:52.707845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:52.824910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:52.825676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.825964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.826244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.855071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:52.856025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:52.857774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:52.865632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:52.867195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:37:52.867989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 54/107] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:52.900660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:52.917183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bc8b8-9aaa-44d5-925b-8138b138e4ae None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:52.958936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ae23e-b3aa-4bfa-a25b-a1c6656fbb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:52.959685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ae23e-b3aa-4bfa-a25b-a1c6656fbb18 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.959974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ae23e-b3aa-4bfa-a25b-a1c6656fbb18 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.960955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ae23e-b3aa-4bfa-a25b-a1c6656fbb18 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.961728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ae23e-b3aa-4bfa-a25b-a1c6656fbb18 None admin] RBAC: Adding query filter params (name=reader) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:52.962037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ae23e-b3aa-4bfa-a25b-a1c6656fbb18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:52.963432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ae23e-b3aa-4bfa-a25b-a1c6656fbb18 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:52.972248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 54/108] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:52.973826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702533df-41d8-484a-9ad3-6beeb91cdc4d None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:52.974173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702533df-41d8-484a-9ad3-6beeb91cdc4d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:52.974456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702533df-41d8-484a-9ad3-6beeb91cdc4d None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:52.978629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 55/109] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:52.982154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702533df-41d8-484a-9ad3-6beeb91cdc4d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.057403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498cac0-ad52-405a-bd95-30260ed1073a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.097745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80682f-abfb-4e76-83ae-e57ac0873307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.099176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80682f-abfb-4e76-83ae-e57ac0873307 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.099551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80682f-abfb-4e76-83ae-e57ac0873307 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.099908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80682f-abfb-4e76-83ae-e57ac0873307 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.100852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80682f-abfb-4e76-83ae-e57ac0873307 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:53.101358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80682f-abfb-4e76-83ae-e57ac0873307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:53.102783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80682f-abfb-4e76-83ae-e57ac0873307 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:53.122761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 55/110] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:53.125184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b89e3-facc-4e3e-bceb-cfb9cc05727f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.129916 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b89e3-facc-4e3e-bceb-cfb9cc05727f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.129916 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b89e3-facc-4e3e-bceb-cfb9cc05727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.135311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702533df-41d8-484a-9ad3-6beeb91cdc4d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.145536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-059b89e3-facc-4e3e-bceb-cfb9cc05727f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:53.146823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-059b89e3-facc-4e3e-bceb-cfb9cc05727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:53.166112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:53.182377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.183625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.184087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.184556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] PATH_INFO: `/v3/users/8989b9983db542e88edbb5c9d6fd2669` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.213595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] RBAC: Authorizing `identity:get_user(user_id=8989b9983db542e88edbb5c9d6fd2669)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:53.216349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:53.218134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 56/111] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:52 2022] GET /identity/v3/users/8989b9983db542e88edbb5c9d6fd2669 => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:53.218906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.231743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:53.259553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 56/112] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:53.260553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b89e3-facc-4e3e-bceb-cfb9cc05727f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.375488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568bc0a-d926-46c0-96b2-a8057ef1c899 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.411255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.412211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.412928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.413256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.414614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:53.416165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:53.418147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 57/113] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:53 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:53.419044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.460859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b89e3-facc-4e3e-bceb-cfb9cc05727f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.470557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:53.525850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a08930-86b1-4a50-9070-9c99217d04d8 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.549655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321f1f8-56ee-46a2-bdc9-fba2107bbf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.550337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f1f8-56ee-46a2-bdc9-fba2107bbf29 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.550635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f1f8-56ee-46a2-bdc9-fba2107bbf29 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.550916 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f1f8-56ee-46a2-bdc9-fba2107bbf29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.563105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.564036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.564330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.564614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.577368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:53.585448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:53.586334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:53.587165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 58/114] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:37:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:53.591901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.688023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:53.700081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321f1f8-56ee-46a2-bdc9-fba2107bbf29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:53.701830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321f1f8-56ee-46a2-bdc9-fba2107bbf29 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:53.706806 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457604-c12d-4a48-b390-b58b6aa67d94 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.716744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:53.742104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.742995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.743284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.743600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.744822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:53.746288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:53.748050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 59/115] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:53 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:53.748815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.814823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 57/116] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:37:53.816353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855e562-ed2f-4f00-852e-5b1e4342e29c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.816652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855e562-ed2f-4f00-852e-5b1e4342e29c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.816932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855e562-ed2f-4f00-852e-5b1e4342e29c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.818356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 58/117] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:53.823584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e9d5-d8a0-4ff2-bf2f-6ac4788bf658 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.823883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e9d5-d8a0-4ff2-bf2f-6ac4788bf658 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.824210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e9d5-d8a0-4ff2-bf2f-6ac4788bf658 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.825658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 59/118] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:53.826248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e9d5-d8a0-4ff2-bf2f-6ac4788bf658 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.862391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d1233-1170-4535-a333-afdee8d1049c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.907719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.908832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.909128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.909411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.910501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:53.910973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:53.912421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:53.925553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 60/119] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:53.925553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:53.929427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:53.941075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e9d5-d8a0-4ff2-bf2f-6ac4788bf658 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:53.977524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c25bed-d383-4970-b8ad-4483def1f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:53.978428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c25bed-d383-4970-b8ad-4483def1f407 None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:53.978702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c25bed-d383-4970-b8ad-4483def1f407 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:53.978975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c25bed-d383-4970-b8ad-4483def1f407 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca/users/1e936b4c92074da5a0a5d5ff77dda5ea/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:53.980977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c25bed-d383-4970-b8ad-4483def1f407 None admin] RBAC: Authorizing `identity:create_grant(project_id=3505cab8f3094ff785667548ec5311ca, user_id=1e936b4c92074da5a0a5d5ff77dda5ea, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:53.982626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c25bed-d383-4970-b8ad-4483def1f407 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:53.985322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:54.000996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 60/120] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:37:53 2022] PUT /identity/v3/projects/3505cab8f3094ff785667548ec5311ca/users/1e936b4c92074da5a0a5d5ff77dda5ea/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:37:54.013706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078643-19c5-4fdc-b5ea-928d3a9ead31 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.014012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078643-19c5-4fdc-b5ea-928d3a9ead31 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.014471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078643-19c5-4fdc-b5ea-928d3a9ead31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.034786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c3078643-19c5-4fdc-b5ea-928d3a9ead31 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:54.036394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c3078643-19c5-4fdc-b5ea-928d3a9ead31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:54.078818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70868b4-3612-470d-a7c3-5404f054d215 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:54.122864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 61/121] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:54.128152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:54.129103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.129400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.129685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.130485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73965a97-b9eb-4e8b-af33-1901eba1bc2e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.131091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:54.131213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73965a97-b9eb-4e8b-af33-1901eba1bc2e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.131840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73965a97-b9eb-4e8b-af33-1901eba1bc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.133001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:54.140473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 61/122] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:53 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:54.141642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:54.168478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-73965a97-b9eb-4e8b-af33-1901eba1bc2e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:54.170073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-73965a97-b9eb-4e8b-af33-1901eba1bc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:54.262090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 62/123] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:54.263728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73965a97-b9eb-4e8b-af33-1901eba1bc2e None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:54.270824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df74b8-3ec4-4a32-9361-0d1aa79e85b4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:54.280270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:54.283029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:54.320206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:54.322025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.322459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.322815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] PATH_INFO: `/v3/users/546aa1cc97e64f3e87a1b21e2088baa8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.324246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] RBAC: Authorizing `identity:get_user(user_id=546aa1cc97e64f3e87a1b21e2088baa8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:54.326699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:54.333175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 62/124] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:54 2022] GET /identity/v3/users/546aa1cc97e64f3e87a1b21e2088baa8 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:54.333175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:54.423865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73965a97-b9eb-4e8b-af33-1901eba1bc2e None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:54.465057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537b623-11ed-4e23-91f7-02e63fcafc66 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:54.467174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:54.468921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.469351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.469795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.470782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] RBAC: Adding query filter params (role.id=2f177b995bd14bb4b85362a4f62b975d, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=8989b9983db542e88edbb5c9d6fd2669) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:54.471244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:54.473369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:54.481411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 63/125] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:37:54 2022] GET /identity/v3/role_assignments?user.id=8989b9983db542e88edbb5c9d6fd2669&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=2f177b995bd14bb4b85362a4f62b975d => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:54.482817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:54.521974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:54.522950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.523245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.523552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.545518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:54.558982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:54.559908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:54.560656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:54.563128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 63/126] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:54.564816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:54.619336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:54.680880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:54.704316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd4bff-87bb-42c1-8465-3d3ba2318d62 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:54.717112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce4610-46c5-4f9a-90d4-939dc28b705d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:54.771706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:54.772729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.772936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.773217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.774646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:54.777018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:54.783440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 64/127] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:54 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:54.790104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:54.796769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:54.797626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.797911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.798187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.811714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:54.821015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:54.821884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:37:54.822705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 64/128] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:37:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:54.823327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:54.922197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279c80a-796d-4f1a-8138-371027dcfb33 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:54.923919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:54.958257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:54.958948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:54.959239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:54.959576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:54.960546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] RBAC: Adding query filter params (name=member) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:54.960863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:54.962270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:54.975551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 65/129] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:54.976469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.070264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ad5b4-a7ab-44f3-933b-51383bd61947 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.080325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bbabc-5a8e-4217-96bb-c1905d47bf95 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.125545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.126534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.127015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.127550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.128506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:55.129030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.130609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.139601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 66/130] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:37:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:55.141148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.171773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:55.179637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.180577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.180868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.181154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca/users/546aa1cc97e64f3e87a1b21e2088baa8/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.188753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] RBAC: Authorizing `identity:create_grant(project_id=3505cab8f3094ff785667548ec5311ca, user_id=546aa1cc97e64f3e87a1b21e2088baa8, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.190356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.200655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:55.243916 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 65/131] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:37:54 2022] PUT /identity/v3/projects/3505cab8f3094ff785667548ec5311ca/users/546aa1cc97e64f3e87a1b21e2088baa8/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:37:55.247823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.264617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d593672-f253-469e-bb1a-74eaa693cc12 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.315923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8913bfd-8c38-4f81-a723-3c9be9de895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.317290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8913bfd-8c38-4f81-a723-3c9be9de895f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.317776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8913bfd-8c38-4f81-a723-3c9be9de895f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.318253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8913bfd-8c38-4f81-a723-3c9be9de895f None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.319991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8913bfd-8c38-4f81-a723-3c9be9de895f None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.321903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8913bfd-8c38-4f81-a723-3c9be9de895f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.323977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 67/132] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:55 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:55.325789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f9e71-5e48-4c64-9788-17b28e3c42f3 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.326295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f9e71-5e48-4c64-9788-17b28e3c42f3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.326795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f9e71-5e48-4c64-9788-17b28e3c42f3 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.328490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 68/133] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:55.330207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f9e71-5e48-4c64-9788-17b28e3c42f3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.382394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95edf8-448c-4d73-9474-ed4ed7815599 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.419360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8afd5-8608-4a9e-93d3-f2c2f0112603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.419812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8afd5-8608-4a9e-93d3-f2c2f0112603 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.420094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8afd5-8608-4a9e-93d3-f2c2f0112603 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.420381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8afd5-8608-4a9e-93d3-f2c2f0112603 None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.421713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8afd5-8608-4a9e-93d3-f2c2f0112603 None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.423245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8afd5-8608-4a9e-93d3-f2c2f0112603 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.425482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 66/134] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:55 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:55.426762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ccbe-4418-411a-b4d4-b60175a5a52f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.427056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ccbe-4418-411a-b4d4-b60175a5a52f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.427341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ccbe-4418-411a-b4d4-b60175a5a52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.449929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-1041ccbe-4418-411a-b4d4-b60175a5a52f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:55.451332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1041ccbe-4418-411a-b4d4-b60175a5a52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:55.463864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f9e71-5e48-4c64-9788-17b28e3c42f3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.468180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:55.551773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:55.556162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 67/135] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:55.556464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.556986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041ccbe-4418-411a-b4d4-b60175a5a52f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.558215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.558698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.559165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] PATH_INFO: `/v3/projects/6e5427facf27427dabca7ffa81326774` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.560747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5427facf27427dabca7ffa81326774)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.562487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.564523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 69/136] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:55 2022] GET /identity/v3/projects/6e5427facf27427dabca7ffa81326774 => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:55.570445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.666238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041ccbe-4418-411a-b4d4-b60175a5a52f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.681363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464a7f4-6cfd-4413-9b2a-3760d13e4345 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.715936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.716852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.717142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.717424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.718747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.720286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.721435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.722232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 68/137] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:55 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:55.722805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.723711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.724289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.724757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] PATH_INFO: `/v3/projects/6e5427facf27427dabca7ffa81326774/users/1e936b4c92074da5a0a5d5ff77dda5ea/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.727641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e5427facf27427dabca7ffa81326774, user_id=1e936b4c92074da5a0a5d5ff77dda5ea, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.729299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.755113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 70/138] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:37:55 2022] PUT /identity/v3/projects/6e5427facf27427dabca7ffa81326774/users/1e936b4c92074da5a0a5d5ff77dda5ea/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:37:55.756817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.799948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:55.801273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:55.853966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04ff3f-f830-484c-a682-acd58d6f0726 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:55.896898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:55.897982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:55.898238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:55.898647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:55.900628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:55.903464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:55.904485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 69/139] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:37:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:55.914402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:55.983528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54a791-d303-41f4-886b-7f122f68b755 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:56.040077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17be28-a38f-415b-a988-b6f970bcb39b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:56.062120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350596-9e36-4277-a828-1090bae3c5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:56.063466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350596-9e36-4277-a828-1090bae3c5db None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:56.063971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350596-9e36-4277-a828-1090bae3c5db None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:56.064455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350596-9e36-4277-a828-1090bae3c5db None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca/users/1e936b4c92074da5a0a5d5ff77dda5ea/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:56.066981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350596-9e36-4277-a828-1090bae3c5db None admin] RBAC: Authorizing `identity:create_grant(project_id=3505cab8f3094ff785667548ec5311ca, user_id=1e936b4c92074da5a0a5d5ff77dda5ea, role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:56.068704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350596-9e36-4277-a828-1090bae3c5db None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:56.074760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:56.084543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a2a03-7ae1-4e91-8354-3cde57742143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:56.085763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a2a03-7ae1-4e91-8354-3cde57742143 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:56.086300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a2a03-7ae1-4e91-8354-3cde57742143 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:56.086723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a2a03-7ae1-4e91-8354-3cde57742143 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:56.087745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a2a03-7ae1-4e91-8354-3cde57742143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:56.090051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a2a03-7ae1-4e91-8354-3cde57742143 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:56.098163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 71/140] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:37:55 2022] PUT /identity/v3/projects/3505cab8f3094ff785667548ec5311ca/users/1e936b4c92074da5a0a5d5ff77dda5ea/roles/20865fed4c894cccb7f1c223e95353ec => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:37:56.128191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 70/141] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:37:55 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:37:56.769051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0e627-b5a5-4c50-8323-fc18db6bc6e9 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:56.769438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0e627-b5a5-4c50-8323-fc18db6bc6e9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:56.769628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0e627-b5a5-4c50-8323-fc18db6bc6e9 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:56.771170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 72/142] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:56.778795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a015-7503-4f73-847c-618c82436f17 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:56.780726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a015-7503-4f73-847c-618c82436f17 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:56.780726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a015-7503-4f73-847c-618c82436f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:56.800217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8777a015-7503-4f73-847c-618c82436f17 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:56.801127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8777a015-7503-4f73-847c-618c82436f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:56.884281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 71/143] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:56.910723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0e627-b5a5-4c50-8323-fc18db6bc6e9 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:57.007043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:57.030456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0e627-b5a5-4c50-8323-fc18db6bc6e9 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:57.069025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:57.069893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:57.070217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:57.070458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:57.084397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:57.092430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:57.093256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:57.094480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 73/144] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:57.099530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777a015-7503-4f73-847c-618c82436f17 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:57.216053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777a015-7503-4f73-847c-618c82436f17 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:57.235167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:57.254255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:57.254947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:57.255235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:57.255745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:57.256546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:57.257015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:57.258862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:57.267687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 72/145] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:57.273671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:57.387827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1b373-91dc-4080-a642-56dc1d16f0f7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:57.423252 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9533a0-6f96-470d-9427-87ac7f2054f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:57.424181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9533a0-6f96-470d-9427-87ac7f2054f2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:57.424461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9533a0-6f96-470d-9427-87ac7f2054f2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:57.424736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9533a0-6f96-470d-9427-87ac7f2054f2 None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:57.426047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9533a0-6f96-470d-9427-87ac7f2054f2 None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:57.427549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9533a0-6f96-470d-9427-87ac7f2054f2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:57.429604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 74/146] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:57 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:57.435445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:57.544487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89fa40-5f4a-4f54-ad30-3e3f55d12b21 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:57.578020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3bc8d-40a9-46bc-b67a-d571686d479f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:57.578350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3bc8d-40a9-46bc-b67a-d571686d479f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:57.578608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3bc8d-40a9-46bc-b67a-d571686d479f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:57.580029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 75/147] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:57.583069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:57.583775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:57.584064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:57.584340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:57.586333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3039a-926e-4a84-aff2-c5c707825f8f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:57.586618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3039a-926e-4a84-aff2-c5c707825f8f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:57.586896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3039a-926e-4a84-aff2-c5c707825f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:57.612875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-fbf3039a-926e-4a84-aff2-c5c707825f8f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:57.614465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fbf3039a-926e-4a84-aff2-c5c707825f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:57.624609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:57.707351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 76/148] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:37:57.711883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:57.754873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:57.756412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:57.757456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3039a-926e-4a84-aff2-c5c707825f8f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:57.898040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3039a-926e-4a84-aff2-c5c707825f8f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:57.951765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 73/149] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:37:57.953730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:57.954772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:57.955259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:57.955746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:57.964818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:57.980548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:57.985950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:57.999407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:58.000295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:37:58.001034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:37:58.003867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 77/150] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:58.009346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:58.115754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:58.125707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17498f-a225-4b53-bfc5-897b05c2d5ff None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:58.174796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:58.175714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:58.176007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:58.176281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:58.177507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:58.179051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:58.184901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 74/151] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:57 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:58.200387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:58.237817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f96675-64e0-4823-89e8-095f9d276094 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:58.293212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:58.293987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:58.294190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:58.294463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:58.295238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] RBAC: Adding query filter params (name=reader) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:58.295577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:58.296983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:58.315722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 78/152] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:58.320648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:58.359768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc488dc-611f-46bd-af36-053436baf871 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:58.410845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:58.411567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:58.411857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:58.412139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:58.413820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=1e936b4c92074da5a0a5d5ff77dda5ea) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:58.414145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:58.415645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:58.426213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 75/153] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:37:58 2022] GET /identity/v3/role_assignments?user.id=1e936b4c92074da5a0a5d5ff77dda5ea&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=fed756b9361747358b9a5948eb9a1442 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:58.451745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:58.525478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5835af-84b4-4f40-a1f6-9c075a8bc841 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:58.586758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:58.587708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:58.587991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:58.588266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] PATH_INFO: `/v3/users/8989b9983db542e88edbb5c9d6fd2669` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:58.592288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] RBAC: Authorizing `identity:get_user(user_id=8989b9983db542e88edbb5c9d6fd2669)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:58.593812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:58.596074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 79/154] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:58 2022] GET /identity/v3/users/8989b9983db542e88edbb5c9d6fd2669 => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:58.601408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:58.625488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:58.746862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac61113-b906-427c-89ef-155ad46c023a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:58.791630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d81296-7873-4bc2-92a5-a20245dca0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:58.792440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81296-7873-4bc2-92a5-a20245dca0e5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:58.792728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81296-7873-4bc2-92a5-a20245dca0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:58.793008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81296-7873-4bc2-92a5-a20245dca0e5 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:58.794302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d81296-7873-4bc2-92a5-a20245dca0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:58.795856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d81296-7873-4bc2-92a5-a20245dca0e5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:58.797690 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 76/155] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:37:58 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:58.803736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:58.889141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3f0b-db64-4159-8e64-3019f3673002 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:58.889476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3f0b-db64-4159-8e64-3019f3673002 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:58.889709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3f0b-db64-4159-8e64-3019f3673002 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:58.891150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 77/156] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:37:58.898415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2d4a-4d67-4df0-a7bc-9da2464dc2a7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:58.898768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2d4a-4d67-4df0-a7bc-9da2464dc2a7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:58.898985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2d4a-4d67-4df0-a7bc-9da2464dc2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:58.925736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d98c2d4a-4d67-4df0-a7bc-9da2464dc2a7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:58.926963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d98c2d4a-4d67-4df0-a7bc-9da2464dc2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:58.995606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf11ac-61a3-46fa-a3a2-53bec6157c02 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:59.031723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 78/157] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:37:59.043817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:59.057189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:59.058120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:59.058409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:59.058743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca/users/8989b9983db542e88edbb5c9d6fd2669/roles/2f177b995bd14bb4b85362a4f62b975d` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:59.063928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] RBAC: Authorizing `identity:create_grant(project_id=3505cab8f3094ff785667548ec5311ca, user_id=8989b9983db542e88edbb5c9d6fd2669, role_id=2f177b995bd14bb4b85362a4f62b975d)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:59.065414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:59.072144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c2d4a-4d67-4df0-a7bc-9da2464dc2a7 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:59.087792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:59.095659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 80/158] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:37:58 2022] PUT /identity/v3/projects/3505cab8f3094ff785667548ec5311ca/users/8989b9983db542e88edbb5c9d6fd2669/roles/2f177b995bd14bb4b85362a4f62b975d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:37:59.215015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c2d4a-4d67-4df0-a7bc-9da2464dc2a7 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:59.254890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9092c-e0ee-44ba-be88-89db65302551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:59.255770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9092c-e0ee-44ba-be88-89db65302551 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:59.256059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9092c-e0ee-44ba-be88-89db65302551 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:59.256337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9092c-e0ee-44ba-be88-89db65302551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:59.270235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9092c-e0ee-44ba-be88-89db65302551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-23d9092c-e0ee-44ba-be88-89db65302551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:59.278501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:59.279492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:37:59.280793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 79/159] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:37:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:37:59.287104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:59.293701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aed174-5676-486c-8758-8cf8a96c6b15 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:59.294045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aed174-5676-486c-8758-8cf8a96c6b15 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:59.294279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aed174-5676-486c-8758-8cf8a96c6b15 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:59.295697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 80/160] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:37:59.302937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd7e84-2c15-471b-808c-3aacb6425155 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:59.303215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd7e84-2c15-471b-808c-3aacb6425155 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:59.303538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd7e84-2c15-471b-808c-3aacb6425155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:59.328572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-dcbd7e84-2c15-471b-808c-3aacb6425155 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:37:59.329815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-dcbd7e84-2c15-471b-808c-3aacb6425155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:37:59.422326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3e9e1-f7d2-4182-b470-8e6057e93f16 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:59.493109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:59.494270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:59.494568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:59.494855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:59.495671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:37:59.495988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:59.497391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:59.507248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 81/161] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:37:59.513413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:59.522045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:59.536756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 81/162] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:37:59.578099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:59.580485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd7e84-2c15-471b-808c-3aacb6425155 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:59.631719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ec47c-f201-4ca0-b9fb-aa38ab2a9042 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:59.705741 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:59.706653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:59.706970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:59.707213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] PATH_INFO: `/v3/users/546aa1cc97e64f3e87a1b21e2088baa8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:59.712673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] RBAC: Authorizing `identity:get_user(user_id=546aa1cc97e64f3e87a1b21e2088baa8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:37:59.714179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:37:59.715912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 82/163] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:37:59 2022] GET /identity/v3/users/546aa1cc97e64f3e87a1b21e2088baa8 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:37:59.721257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:59.776878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd7e84-2c15-471b-808c-3aacb6425155 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:59.843675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:37:59.844419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:37:59.844701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:37:59.844975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:37:59.865303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:37:59.873290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:37:59.874133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:37:59.875346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 82/164] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:37:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:37:59.880568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:37:59.941499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde2dfd-d476-42da-b6b7-44eb121780fe None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:59.951801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:37:59.982160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0bf9e-0723-46d1-a672-3a5bd5af9cd9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:37:59.991393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:00.019635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:00.020522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:00.020976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:00.021355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:00.027407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:00.027407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:00.027407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:00.033021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 83/165] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:37:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:00.034955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:00.035807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:00.036247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:00.036659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:00.040366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:00.202973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8355ff5-b176-4500-9d6f-88a7139ee76a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:00.232493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:00.234271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:00.239595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:00.268964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:00.269877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:00.270158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:00.270429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:00.271668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:00.274085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:00.275941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 84/166] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:00 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:00.281494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:00.440185 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 83/167] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 12:38:00.454229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:00.464660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c711d5-e54f-41c5-b49f-b42272654b87 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:00.495245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:00.535998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:00.536670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:00.536952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:00.537222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:00.648907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:00.654666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548e349-cef8-42f6-8368-78fdabb886fc None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:00.674117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:00.675660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:00.714773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:00.715593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:00.715874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:00.716146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:00.717346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:00.718810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:00.720873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 84/168] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:00 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:00.727635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:00.853203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 85/169] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 12:38:00.859162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:00.961997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:00.982739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350745d-9fab-4e52-aa74-6c68cdef217e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.029747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:01.051210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6210-f7f9-49c0-8788-46608a287146 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.055255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f72a-a850-4507-b732-3cac0a19b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:01.055966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f72a-a850-4507-b732-3cac0a19b47c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.056250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f72a-a850-4507-b732-3cac0a19b47c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.056529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f72a-a850-4507-b732-3cac0a19b47c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.057392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f72a-a850-4507-b732-3cac0a19b47c None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=546aa1cc97e64f3e87a1b21e2088baa8) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:01.057716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f72a-a850-4507-b732-3cac0a19b47c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:01.059170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f72a-a850-4507-b732-3cac0a19b47c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:01.074443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 85/170] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:00 2022] GET /identity/v3/role_assignments?user.id=546aa1cc97e64f3e87a1b21e2088baa8&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:01.075786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db03f-0016-4abf-8663-b9c9df68b5bb None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.076081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db03f-0016-4abf-8663-b9c9df68b5bb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.076360 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db03f-0016-4abf-8663-b9c9df68b5bb None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.077611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 86/171] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:01.078703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b7deb-4cec-4012-b620-268a43997782 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.078986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b7deb-4cec-4012-b620-268a43997782 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.079387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b7deb-4cec-4012-b620-268a43997782 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.080843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 87/172] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:01.086497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de94ab4-40ce-45d4-81a7-04ec9e4cfdc6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.086684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de94ab4-40ce-45d4-81a7-04ec9e4cfdc6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.086965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de94ab4-40ce-45d4-81a7-04ec9e4cfdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.116673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3de94ab4-40ce-45d4-81a7-04ec9e4cfdc6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:01.116673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3de94ab4-40ce-45d4-81a7-04ec9e4cfdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:01.142791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eada17-42b4-48b0-ae6b-f3bc679f6b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:01.143738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eada17-42b4-48b0-ae6b-f3bc679f6b16 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.144027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eada17-42b4-48b0-ae6b-f3bc679f6b16 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.144312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eada17-42b4-48b0-ae6b-f3bc679f6b16 None admin] PATH_INFO: `/v3/projects/6e5427facf27427dabca7ffa81326774` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.147478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eada17-42b4-48b0-ae6b-f3bc679f6b16 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5427facf27427dabca7ffa81326774)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:01.149176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eada17-42b4-48b0-ae6b-f3bc679f6b16 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:01.151247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 86/173] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:00 2022] GET /identity/v3/projects/6e5427facf27427dabca7ffa81326774 => generated 312 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:01.152271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f90b-2309-4924-9b98-18eb12299bef None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.152558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f90b-2309-4924-9b98-18eb12299bef None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.152839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f90b-2309-4924-9b98-18eb12299bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.196445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8b46f90b-2309-4924-9b98-18eb12299bef None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:01.197723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8b46f90b-2309-4924-9b98-18eb12299bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:01.271020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 88/174] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:01.273147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2fd8-248e-41de-9ef3-78990d3655fc None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.273448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2fd8-248e-41de-9ef3-78990d3655fc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.273733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2fd8-248e-41de-9ef3-78990d3655fc None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.275336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 89/175] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:01.276074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd2fd8-248e-41de-9ef3-78990d3655fc None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:01.298767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 87/176] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:01.304566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c9e8-40cb-4160-82da-15101725a1d7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.304859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c9e8-40cb-4160-82da-15101725a1d7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.305259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c9e8-40cb-4160-82da-15101725a1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.336329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-2359c9e8-40cb-4160-82da-15101725a1d7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:01.337635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2359c9e8-40cb-4160-82da-15101725a1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:01.385855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:01.414092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd2fd8-248e-41de-9ef3-78990d3655fc None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.420574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 88/177] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:01.421442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c9e8-40cb-4160-82da-15101725a1d7 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:01.440267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:01.474437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:01.475146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.475437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.475782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.476982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=6e5427facf27427dabca7ffa81326774, user.id=1e936b4c92074da5a0a5d5ff77dda5ea) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:01.477309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:01.478787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:01.486366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 90/178] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:01 2022] GET /identity/v3/role_assignments?user.id=1e936b4c92074da5a0a5d5ff77dda5ea&scope.project.id=6e5427facf27427dabca7ffa81326774&role.id=fed756b9361747358b9a5948eb9a1442 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:01.487082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:01.546529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c9e8-40cb-4160-82da-15101725a1d7 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.587715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:01.589151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.589646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.590161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.603405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:01.612957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:01.613879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:01.614637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:01.622979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 89/179] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:01.624386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:01.713836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:01.742738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927b691-6ae7-4856-b080-3fba636acbf2 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.775202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71d631-7d65-4258-bb03-f3d991ad8f35 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.789130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:01.790226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.790763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.791139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.804635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:01.814973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:01.815911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:01.816701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:01.817302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:01.817302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.817302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.817302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.817561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 90/180] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:01.818498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:01.832009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:01.843580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:01.844501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:01.845288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 91/181] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:01.846788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:01.865241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:01.944008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d9d94-7c6f-4a82-84d9-46b65f1be972 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.976393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753afea-2688-4ea8-a520-1a4ff61e8462 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:01.991155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:01.992515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:01.992802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:01.993081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:01.993840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:01.994148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:01.995618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:01.997815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.005117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 91/182] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:02.005833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.016186 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba25ee-595d-4e2a-87b4-8805cfcfba1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.016871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba25ee-595d-4e2a-87b4-8805cfcfba1f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.017155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba25ee-595d-4e2a-87b4-8805cfcfba1f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.017432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba25ee-595d-4e2a-87b4-8805cfcfba1f None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.018568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba25ee-595d-4e2a-87b4-8805cfcfba1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:02.018879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba25ee-595d-4e2a-87b4-8805cfcfba1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.020594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba25ee-595d-4e2a-87b4-8805cfcfba1f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.029726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 92/183] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:02.035407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321a011-4c6c-4162-b90a-05d7731c0e27 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.035696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321a011-4c6c-4162-b90a-05d7731c0e27 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.035978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321a011-4c6c-4162-b90a-05d7731c0e27 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.037390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 93/184] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:02.038402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321a011-4c6c-4162-b90a-05d7731c0e27 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.107648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.133129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a902-bd0c-4b4d-8ae8-f27a2e40cb3d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.150069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321a011-4c6c-4162-b90a-05d7731c0e27 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.169176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.170099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.170613 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.170892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.171668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:02.172049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.173487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.186201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 92/185] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:02.186201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.186416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444125bf-fbc8-4718-80c8-02e20274b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.186505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444125bf-fbc8-4718-80c8-02e20274b65b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.186822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444125bf-fbc8-4718-80c8-02e20274b65b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.187104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444125bf-fbc8-4718-80c8-02e20274b65b None admin] PATH_INFO: `/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.188355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444125bf-fbc8-4718-80c8-02e20274b65b None admin] RBAC: Authorizing `identity:get_user(user_id=1e936b4c92074da5a0a5d5ff77dda5ea)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.189816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444125bf-fbc8-4718-80c8-02e20274b65b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.191461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 94/186] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/users/1e936b4c92074da5a0a5d5ff77dda5ea => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:02.192706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c874a1-e88d-4328-aa98-9e4f386d5556 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.192993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c874a1-e88d-4328-aa98-9e4f386d5556 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.193265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c874a1-e88d-4328-aa98-9e4f386d5556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.210949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d3c874a1-e88d-4328-aa98-9e4f386d5556 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:02.212181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d3c874a1-e88d-4328-aa98-9e4f386d5556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:02.250463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.279139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 95/187] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:02.279139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c874a1-e88d-4328-aa98-9e4f386d5556 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.304959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efcfe4-2728-482e-978e-c863c2483771 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.342021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.342928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.343215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.343520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] PATH_INFO: `/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.360023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.360983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] RBAC: Authorizing `identity:get_user(user_id=0bca04d6a1cb4f8d9bce3519d1e55515)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.362783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.369935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 93/188] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:02.369935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.404398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c874a1-e88d-4328-aa98-9e4f386d5556 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.440236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.441121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.441407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.441681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] PATH_INFO: `/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.442857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] RBAC: Authorizing `identity:get_user(user_id=0bca04d6a1cb4f8d9bce3519d1e55515)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.444338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.446101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 96/189] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:02.446847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.469226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f42aff-12df-4d2c-979b-460ec38c9223 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.485887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.506863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb32db7-6af3-4e42-bfe0-8f9aa6efaef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.507597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb32db7-6af3-4e42-bfe0-8f9aa6efaef7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.507904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb32db7-6af3-4e42-bfe0-8f9aa6efaef7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.508155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb32db7-6af3-4e42-bfe0-8f9aa6efaef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.545627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcab85-bb77-482c-b9cb-c73b8e2a302f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.573653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb32db7-6af3-4e42-bfe0-8f9aa6efaef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.575141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb32db7-6af3-4e42-bfe0-8f9aa6efaef7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.578270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.579082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.579386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.579675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.580904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.582357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.584055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 97/190] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:02.584810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.587555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.677031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 94/191] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:02.678279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a291098-2e69-4619-8412-ca09bd2a6c4d None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.678538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a291098-2e69-4619-8412-ca09bd2a6c4d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.678850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a291098-2e69-4619-8412-ca09bd2a6c4d None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.680223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 95/192] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:02.680908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a291098-2e69-4619-8412-ca09bd2a6c4d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.689345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38a747-75d6-4355-8e51-b9475308635a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.700367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.724224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.724898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.725183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.725457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.783573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a291098-2e69-4619-8412-ca09bd2a6c4d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:02.792631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.794102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.795943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.819928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:02.820716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:02.820997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:02.821293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:02.887770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:02.889269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:02.898665 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 98/193] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:02.899706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:02.906759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:02.989527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 96/194] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:02.990263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.000294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098d306-7ad9-429f-9ff7-22c5b3a50ef8 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.001813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:03.033738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deca52a-fc16-46e1-9c0d-52e74589c6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.034410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deca52a-fc16-46e1-9c0d-52e74589c6d6 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.034687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deca52a-fc16-46e1-9c0d-52e74589c6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.034957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deca52a-fc16-46e1-9c0d-52e74589c6d6 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.035697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deca52a-fc16-46e1-9c0d-52e74589c6d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.037137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deca52a-fc16-46e1-9c0d-52e74589c6d6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.063792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 99/195] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:02 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:03.065038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f14282-3ee4-48eb-a0f4-05ee76d08f73 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.065324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f14282-3ee4-48eb-a0f4-05ee76d08f73 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.065598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f14282-3ee4-48eb-a0f4-05ee76d08f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.082897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c5f14282-3ee4-48eb-a0f4-05ee76d08f73 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:03.084181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c5f14282-3ee4-48eb-a0f4-05ee76d08f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:03.109240 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d84507-ce72-47a9-85cc-cf83a6b9a04c None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.135658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:03.157054 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.158168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.158455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.158743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.160120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.161839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.164347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 97/196] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:02 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:03.165280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.169922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 100/197] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:03.184512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f14282-3ee4-48eb-a0f4-05ee76d08f73 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.282893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8f6c0-36fb-4d4d-8bbd-c77e0ed18518 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.297709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:03.311220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f14282-3ee4-48eb-a0f4-05ee76d08f73 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.328729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.329622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.329908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.330188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.331487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.332962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.334687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 98/198] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:03.335566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.350215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.351219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.351533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.351808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.353031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.354504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.356588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 101/199] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:03.357367 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.403762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:03.454037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf242-4017-40f5-b46d-68bd7ad5ee3f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.489429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.490072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.490354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.490633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.491709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] RBAC: Adding query filter params (role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=1e936b4c92074da5a0a5d5ff77dda5ea) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:03.491839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed09a2-d931-4d25-be8e-204545612646 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.491918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.493650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.500524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 99/200] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/role_assignments?user.id=1e936b4c92074da5a0a5d5ff77dda5ea&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=20865fed4c894cccb7f1c223e95353ec => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:03.501362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.534934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.535730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.536015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.536292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.566915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:03.575868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:03.576826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:03.577582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 102/201] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:03.578918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.618594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:03.665513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc68776-31e6-4461-b47d-793be579d466 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.705100 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:03.720748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.721585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.721968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.722346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.723311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] RBAC: Adding query filter params (role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=0bca04d6a1cb4f8d9bce3519d1e55515) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:03.723799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.725366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.732382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 100/202] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/role_assignments?user.id=0bca04d6a1cb4f8d9bce3519d1e55515&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=20865fed4c894cccb7f1c223e95353ec => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:03.733431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:03.760271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df9b9-768a-4093-baeb-a078e1b7106f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.832225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.832830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.833117 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.833394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.834262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=0bca04d6a1cb4f8d9bce3519d1e55515) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:03.834734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.836548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.842951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 103/203] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/role_assignments?user.id=0bca04d6a1cb4f8d9bce3519d1e55515&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:03.845431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f430a-4615-4cba-9408-11c0530f4b68 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:03.887786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:03.888635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:03.888982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:03.889325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:03.890200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] RBAC: Adding query filter params (name=reader) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:03.890578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:03.892147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:03.902988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 101/204] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:03.912595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:04.038606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:04.060306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500d6f8-756b-4ef8-87c5-52fe2079b45a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:04.100048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:04.100930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:04.101226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:04.101509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] PATH_INFO: `/v3/users/8989b9983db542e88edbb5c9d6fd2669` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:04.102780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] RBAC: Authorizing `identity:get_user(user_id=8989b9983db542e88edbb5c9d6fd2669)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:04.104347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:04.106071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 104/205] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:03 2022] GET /identity/v3/users/8989b9983db542e88edbb5c9d6fd2669 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:04.111883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:04.214042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:04.284132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f31cf9-c23d-4a9b-9233-766bea7ec085 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:04.319172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:04.320474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:04.320773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:04.321049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:04.403566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:04.404964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:04.464274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:04.536971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 102/206] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:04.542827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:04.719089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15183-3a94-40c9-ac88-d80b973b1867 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:04.768148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38894109-dcbe-4b46-9242-2b473cbbddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:04.769039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38894109-dcbe-4b46-9242-2b473cbbddf7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:04.769348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38894109-dcbe-4b46-9242-2b473cbbddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:04.769598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38894109-dcbe-4b46-9242-2b473cbbddf7 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:04.770830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38894109-dcbe-4b46-9242-2b473cbbddf7 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:04.772316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38894109-dcbe-4b46-9242-2b473cbbddf7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:04.774543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 105/207] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:04 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:04.780437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:04.920225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7159b4-c2a3-4b1d-8cf2-6dae1c6e7aa9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:04.964971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf6798-32ea-4494-94f3-875c422ebd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:04.965641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6798-32ea-4494-94f3-875c422ebd84 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:04.965921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6798-32ea-4494-94f3-875c422ebd84 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:04.966195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6798-32ea-4494-94f3-875c422ebd84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:04.967040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6798-32ea-4494-94f3-875c422ebd84 None admin] RBAC: Adding query filter params (role.id=2f177b995bd14bb4b85362a4f62b975d, scope.project.id=3505cab8f3094ff785667548ec5311ca, user.id=8989b9983db542e88edbb5c9d6fd2669) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:04.967352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6798-32ea-4494-94f3-875c422ebd84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:04.968826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6798-32ea-4494-94f3-875c422ebd84 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:04.978358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 103/208] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:04 2022] GET /identity/v3/role_assignments?user.id=8989b9983db542e88edbb5c9d6fd2669&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=2f177b995bd14bb4b85362a4f62b975d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:05.730131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c24d0-2364-4985-9e46-6f960163182f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:05.730872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c24d0-2364-4985-9e46-6f960163182f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:05.730872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c24d0-2364-4985-9e46-6f960163182f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:05.732139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 106/209] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:05.737742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572aa30-9a0a-4900-8eae-5619b84c3fe3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:05.738291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572aa30-9a0a-4900-8eae-5619b84c3fe3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:05.738291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572aa30-9a0a-4900-8eae-5619b84c3fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:05.755814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-f572aa30-9a0a-4900-8eae-5619b84c3fe3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:05.757051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f572aa30-9a0a-4900-8eae-5619b84c3fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:05.759435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:05.837888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 104/210] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:05.839513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca33cf-459c-4560-b94c-7601f528b5ee None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:05.839807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca33cf-459c-4560-b94c-7601f528b5ee None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:05.840086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca33cf-459c-4560-b94c-7601f528b5ee None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:05.841718 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 107/211] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:05.848915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee6fb6-054f-418d-8b65-cf0a0416bbb3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:05.849183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee6fb6-054f-418d-8b65-cf0a0416bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:05.849473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee6fb6-054f-418d-8b65-cf0a0416bbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:05.862717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca33cf-459c-4560-b94c-7601f528b5ee None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:05.866955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e0ee6fb6-054f-418d-8b65-cf0a0416bbb3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:05.868169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e0ee6fb6-054f-418d-8b65-cf0a0416bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:05.891858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:05.952183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 105/212] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:05.975447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee6fb6-054f-418d-8b65-cf0a0416bbb3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:05.991463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca33cf-459c-4560-b94c-7601f528b5ee None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.028201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.029015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.029293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.029564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.030754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:06.032246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:06.034174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 108/213] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:06.040533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:06.061249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:06.110379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee6fb6-054f-418d-8b65-cf0a0416bbb3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.146410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:06.149925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.150817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.151526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.151526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.164494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:06.165095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a0551-44bb-4efe-ac49-99400fc9ee11 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:06.172779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:06.173909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:06.174917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:06.176087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 106/214] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:06.181604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:06.204089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24456e1a-45c3-4ae9-a09a-de40b707e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.205272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456e1a-45c3-4ae9-a09a-de40b707e831 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.205797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456e1a-45c3-4ae9-a09a-de40b707e831 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.206268 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456e1a-45c3-4ae9-a09a-de40b707e831 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.207190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24456e1a-45c3-4ae9-a09a-de40b707e831 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:06.209019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24456e1a-45c3-4ae9-a09a-de40b707e831 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:06.234508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 109/215] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:06 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:06.300746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976e729-26e1-4b81-a42c-7c0240243891 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.333164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:06.341405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.342749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.343337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.344071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.344903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:06.345212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:06.346727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:06.354123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b77115-41c1-4bac-8dcc-c665f83fbf5c None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.354813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b77115-41c1-4bac-8dcc-c665f83fbf5c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.355314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b77115-41c1-4bac-8dcc-c665f83fbf5c None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.357160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 110/216] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:06.359455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 107/217] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:06.363923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372d76c-b29c-4a11-b474-8ac243c947bb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.364317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372d76c-b29c-4a11-b474-8ac243c947bb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.364699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372d76c-b29c-4a11-b474-8ac243c947bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.366611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:06.384582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1372d76c-b29c-4a11-b474-8ac243c947bb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:06.386151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1372d76c-b29c-4a11-b474-8ac243c947bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:06.464749 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 111/218] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:06.466105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c21e9c-66a4-4ea3-bcde-941c7cf9cbbc None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.466399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c21e9c-66a4-4ea3-bcde-941c7cf9cbbc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.466679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c21e9c-66a4-4ea3-bcde-941c7cf9cbbc None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.468467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 112/219] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:06.473835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2eee6-2db7-4b9e-9405-2a2d64bfc463 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.474138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2eee6-2db7-4b9e-9405-2a2d64bfc463 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.474412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2eee6-2db7-4b9e-9405-2a2d64bfc463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.481353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3b2b-488a-4bfe-b41c-373da14f7393 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.493675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a6f2eee6-2db7-4b9e-9405-2a2d64bfc463 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:06.494846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a6f2eee6-2db7-4b9e-9405-2a2d64bfc463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:06.524060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.525155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.525344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.526877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] PATH_INFO: `/v3/users/acb28d4482ba46b99bc10057d74786d3` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.545217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] RBAC: Authorizing `identity:get_user(user_id=acb28d4482ba46b99bc10057d74786d3)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:06.547286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:06.549236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 108/220] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:06 2022] GET /identity/v3/users/acb28d4482ba46b99bc10057d74786d3 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:06.549548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:06.586612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:06.592551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 113/221] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:06.593973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2eee6-2db7-4b9e-9405-2a2d64bfc463 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:06.621293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:06.677570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7078cd-a5df-44a3-bd5a-9b0f5a85b720 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.716476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.716717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2eee6-2db7-4b9e-9405-2a2d64bfc463 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.717773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.718246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.718706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.730499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:06.738407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:06.739793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:06.740782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:06.743667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 109/222] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:06.744633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:06.764563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.765884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.766412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.766906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.848323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:06.853421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:06.855255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:06.858407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976a2e0-cbce-46bc-9611-953f6a0b7405 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:06.884009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:06.903555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:06.905551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:06.906030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:06.906499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:06.921325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:06.933639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:06.934596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:06.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:06.936735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 110/223] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:06.938349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:06.986698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 114/224] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:06.988191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.073383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cb49f-9780-4357-a93a-3a1bc74fe546 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.100814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:07.106712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f5754-ebf5-4147-8f7e-312aba39aaea None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.119214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.119929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.120209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.120484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.121421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:07.121585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.122957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.133601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 111/225] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.134621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.158815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.160137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.160875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.161373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.162965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:07.164825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.167105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.167779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:07.178482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 115/226] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.180151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.245846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdd7b6-b550-433d-89b1-c6edb21fa89d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.286667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.288654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.289706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.290126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.291956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.293846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.296474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 112/227] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:07 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.297384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.305341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f743e6-9c03-4779-a42b-fdb46c3e5681 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.341641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.342331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.342704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.343099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] PATH_INFO: `/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.344472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] RBAC: Authorizing `identity:get_user(user_id=0bca04d6a1cb4f8d9bce3519d1e55515)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.346017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.348220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 116/228] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:07 2022] GET /identity/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.349582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.382095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:07.412565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:07.418735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b0db-02bb-40dc-a497-f5c58b22a571 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.454527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.455703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.456079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.456448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] PATH_INFO: `/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.457727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] RBAC: Authorizing `identity:get_user(user_id=0bca04d6a1cb4f8d9bce3519d1e55515)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.459506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.461770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 113/229] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:07 2022] GET /identity/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.462864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.469257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78485562-231f-4979-94b4-6b441c466e8c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.512050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.513029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.513455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.513870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.514886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=acb28d4482ba46b99bc10057d74786d3) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:07.515406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.517150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.523913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 117/230] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:07 2022] GET /identity/v3/role_assignments?user.id=acb28d4482ba46b99bc10057d74786d3&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=fed756b9361747358b9a5948eb9a1442 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.525121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.579988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b801e71-3fcd-49e4-be88-2e14075b0120 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.618661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.619676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.619957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.620231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.621588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.623102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.625026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 114/231] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:07 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.631414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.657757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:07.674869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e02cf-d367-4867-9ef3-583d6a87b5d3 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.714844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.715796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.716086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.716357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.717712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.719187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.721496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 118/232] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:07 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:07.728220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:07.742005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:07.822229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff09a17-6e47-4ac1-a95e-c494c4b95526 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.861869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a846e85-81c3-41e5-af38-6ba49e93a3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.862837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a846e85-81c3-41e5-af38-6ba49e93a3be None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.863118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a846e85-81c3-41e5-af38-6ba49e93a3be None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.864775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a846e85-81c3-41e5-af38-6ba49e93a3be None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/0bca04d6a1cb4f8d9bce3519d1e55515/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.872216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a846e85-81c3-41e5-af38-6ba49e93a3be None admin] RBAC: Authorizing `identity:create_grant(project_id=cd408398a4fd4a1d970960d255e9e000, user_id=0bca04d6a1cb4f8d9bce3519d1e55515, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.873893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a846e85-81c3-41e5-af38-6ba49e93a3be None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.902857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 115/233] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:07 2022] PUT /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/0bca04d6a1cb4f8d9bce3519d1e55515/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:07.937077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9105-a0c0-4d80-aad5-7192b15e5505 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:07.975192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3dc7-6e5a-40d0-a7ee-6509bb09cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:07.976258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3dc7-6e5a-40d0-a7ee-6509bb09cbb7 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:07.976541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3dc7-6e5a-40d0-a7ee-6509bb09cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:07.976814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3dc7-6e5a-40d0-a7ee-6509bb09cbb7 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/0bca04d6a1cb4f8d9bce3519d1e55515/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:07.979334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3dc7-6e5a-40d0-a7ee-6509bb09cbb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd408398a4fd4a1d970960d255e9e000, user_id=0bca04d6a1cb4f8d9bce3519d1e55515, role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:07.980825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3dc7-6e5a-40d0-a7ee-6509bb09cbb7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:07.996886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 119/234] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:07 2022] PUT /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/0bca04d6a1cb4f8d9bce3519d1e55515/roles/20865fed4c894cccb7f1c223e95353ec => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:08.770352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562528bf-8719-4651-a0bf-194919facb97 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:08.770878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562528bf-8719-4651-a0bf-194919facb97 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:08.770954 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562528bf-8719-4651-a0bf-194919facb97 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:08.772597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 116/235] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:08.790285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b30e97-a635-4875-8cc0-408402174d77 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:08.791239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b30e97-a635-4875-8cc0-408402174d77 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:08.791733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b30e97-a635-4875-8cc0-408402174d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:08.816514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-75b30e97-a635-4875-8cc0-408402174d77 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:08.818202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-75b30e97-a635-4875-8cc0-408402174d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:08.837628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:08.863659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55fda-fa61-43ef-9851-5d9ff6af731e None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:08.864291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55fda-fa61-43ef-9851-5d9ff6af731e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:08.864797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55fda-fa61-43ef-9851-5d9ff6af731e None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:08.866691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 117/236] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:08.872360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968db3-3231-4900-8493-f4d9149e5784 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:08.872959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968db3-3231-4900-8493-f4d9149e5784 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:08.873436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968db3-3231-4900-8493-f4d9149e5784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:08.902381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-f0968db3-3231-4900-8493-f4d9149e5784 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:08.903698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f0968db3-3231-4900-8493-f4d9149e5784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:08.907743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 120/237] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:08.949498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b30e97-a635-4875-8cc0-408402174d77 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:08.982515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 118/238] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:09.008238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0968db3-3231-4900-8493-f4d9149e5784 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:09.033340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:09.115530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0968db3-3231-4900-8493-f4d9149e5784 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:09.153797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:09.155019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:09.155534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:09.156007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:09.157542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:09.159316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:09.161570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 119/239] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:09.167530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:09.171307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b30e97-a635-4875-8cc0-408402174d77 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:09.248253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:09.249124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:09.249379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:09.249656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:09.269959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:09.279204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8279a-0a38-41d8-a166-affb4d47ade7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:09.288780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:09.304667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:09.305572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:09.306357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 121/240] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:09.313206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:09.319711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:09.320772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:09.321257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:09.321729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:09.322659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:09.324626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:09.339715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:09.362576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 120/241] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:09 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:38:09.488504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd9e78-0443-467d-98f4-11e4783be610 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:09.525797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:09.526925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:09.527482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:09.527960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:09.528927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] RBAC: Adding query filter params (name=reader) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:09.529430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:09.531063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:09.541938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 122/242] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:09.548124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:09.672990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde95c9-aaad-465f-a5b5-7ad92c3c30c1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:09.712405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007d00e-b4c6-49b1-b264-394d7e7d5313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:09.713310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007d00e-b4c6-49b1-b264-394d7e7d5313 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:09.713598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007d00e-b4c6-49b1-b264-394d7e7d5313 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:09.713883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007d00e-b4c6-49b1-b264-394d7e7d5313 None admin] PATH_INFO: `/v3/users/4831118709fe4e5db8d82bce71fdd83c` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:09.728160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007d00e-b4c6-49b1-b264-394d7e7d5313 None admin] RBAC: Authorizing `identity:get_user(user_id=4831118709fe4e5db8d82bce71fdd83c)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:09.729642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007d00e-b4c6-49b1-b264-394d7e7d5313 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:09.731391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 121/243] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:09 2022] GET /identity/v3/users/4831118709fe4e5db8d82bce71fdd83c => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:09.737016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:09.852055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:09.870191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a52116-371d-436d-9b36-796d98ec9ece None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:09.904813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:09.905921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:09.906433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:09.906906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:09.986927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:09.989279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:10.004705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436f4c-e0bb-4601-ae0b-143663380097 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.005232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436f4c-e0bb-4601-ae0b-143663380097 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.005232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436f4c-e0bb-4601-ae0b-143663380097 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.006644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 122/244] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:10.013181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2d294-f537-4da6-9095-8c44ca32313f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.013455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2d294-f537-4da6-9095-8c44ca32313f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.013741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2d294-f537-4da6-9095-8c44ca32313f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.030622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:10.039973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e2b2d294-f537-4da6-9095-8c44ca32313f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:10.041259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e2b2d294-f537-4da6-9095-8c44ca32313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:10.069396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:10.096707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 123/245] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:10.102604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:10.127742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 123/246] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:10.151214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2d294-f537-4da6-9095-8c44ca32313f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:10.210858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cb7ca-cb82-4b5c-b448-aadccc161657 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:10.246041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:10.246977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.247266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.247563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.248856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:10.250317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:10.252077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 124/247] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:10.258472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:10.266511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2d294-f537-4da6-9095-8c44ca32313f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:10.303625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:10.309090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:10.310449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.310947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.311462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.314972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:10.326466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:10.337174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:10.338071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:10.338838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:10.342565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 124/248] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:10.348871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:10.364800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256a87f-3c30-4f9c-ba6c-99a40d4d6a65 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:10.399557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:10.400276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.400623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.400966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.401885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] RBAC: Adding query filter params (role.id=2f177b995bd14bb4b85362a4f62b975d, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=4831118709fe4e5db8d82bce71fdd83c) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:10.402271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:10.403892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:10.409848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 125/249] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/role_assignments?user.id=4831118709fe4e5db8d82bce71fdd83c&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=2f177b995bd14bb4b85362a4f62b975d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:10.467190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b91a9-4a28-4bb4-853d-03424a134f9a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:10.506880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc12eb-6f32-4836-a982-68ade245fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:10.508108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc12eb-6f32-4836-a982-68ade245fa68 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.508599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc12eb-6f32-4836-a982-68ade245fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.509053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc12eb-6f32-4836-a982-68ade245fa68 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.510022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc12eb-6f32-4836-a982-68ade245fa68 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:10.510529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc12eb-6f32-4836-a982-68ade245fa68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:10.512269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc12eb-6f32-4836-a982-68ade245fa68 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:10.522280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 125/250] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:10.528302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:10.605859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52a82-5185-4748-ac6b-6765ce6154c4 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.606521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52a82-5185-4748-ac6b-6765ce6154c4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.607011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52a82-5185-4748-ac6b-6765ce6154c4 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.608669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 126/251] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:10.614944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46385502-30da-4b60-afb1-f568929a1406 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.615534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46385502-30da-4b60-afb1-f568929a1406 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.616013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46385502-30da-4b60-afb1-f568929a1406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.656441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-46385502-30da-4b60-afb1-f568929a1406 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:10.658066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-46385502-30da-4b60-afb1-f568929a1406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:10.725681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:10.743493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 127/252] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:10.751417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301c397-e3fc-4cfe-9b59-ccfb81a10a38 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:10.759580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:10.777807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46385502-30da-4b60-afb1-f568929a1406 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:10.797455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:10.798339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.798616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.798884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] PATH_INFO: `/v3/users/acb28d4482ba46b99bc10057d74786d3` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.804163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] RBAC: Authorizing `identity:get_user(user_id=acb28d4482ba46b99bc10057d74786d3)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:10.805896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:10.808184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 126/253] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/users/acb28d4482ba46b99bc10057d74786d3 => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:10.813810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:10.891650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46385502-30da-4b60-afb1-f568929a1406 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:10.931471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:10.932650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:10.933133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:10.933605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:10.946076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:10.954944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:10.955834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:10.956610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:10.959644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 128/254] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:10.968229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:11.027892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:11.060760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e5319-7fb8-4864-977c-c17ddd25925e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:11.117512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d881ace-e6f2-424a-b1f9-24ffa4201721 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:11.155048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:11.155803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:11.156096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:11.156651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:11.157424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:11.157737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:11.159146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:11.160160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:11.161151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:11.161503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:11.161869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:11.163278 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:11.169625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 129/255] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:11.172348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:11.174307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 127/256] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:10 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:11.175245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:11.196746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:11.291776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2596095-af33-4e54-ac2c-03970e876608 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:11.304028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:11.319745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:11.332823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158398c-4faf-484b-87ae-fbb27ba45c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:11.334495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158398c-4faf-484b-87ae-fbb27ba45c50 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:11.334980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158398c-4faf-484b-87ae-fbb27ba45c50 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:11.335471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158398c-4faf-484b-87ae-fbb27ba45c50 None admin] PATH_INFO: `/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:11.337767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158398c-4faf-484b-87ae-fbb27ba45c50 None admin] RBAC: Authorizing `identity:get_user(user_id=0bca04d6a1cb4f8d9bce3519d1e55515)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:11.339909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158398c-4faf-484b-87ae-fbb27ba45c50 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:11.346369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 130/257] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:11 2022] GET /identity/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:11.349522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9fb89-2843-489b-81f0-3b59ff9ebca3 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:11.349812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9fb89-2843-489b-81f0-3b59ff9ebca3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:11.350085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9fb89-2843-489b-81f0-3b59ff9ebca3 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:11.352275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 131/258] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:11.353231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9fb89-2843-489b-81f0-3b59ff9ebca3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:11.467405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9fb89-2843-489b-81f0-3b59ff9ebca3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:11.502123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a8806-dd74-4ab6-8d41-30344947b104 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:11.503768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:11.504458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:11.504716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:11.504987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:11.553875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:11.578774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:11.580603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:11.608588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65c88a-8818-4087-bf35-7d8d0ef936ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:11.609512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65c88a-8818-4087-bf35-7d8d0ef936ec None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:11.609796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65c88a-8818-4087-bf35-7d8d0ef936ec None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:11.610071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65c88a-8818-4087-bf35-7d8d0ef936ec None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/acb28d4482ba46b99bc10057d74786d3/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:11.612078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65c88a-8818-4087-bf35-7d8d0ef936ec None admin] RBAC: Authorizing `identity:create_grant(project_id=cd408398a4fd4a1d970960d255e9e000, user_id=acb28d4482ba46b99bc10057d74786d3, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:11.613526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65c88a-8818-4087-bf35-7d8d0ef936ec None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:11.650519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 128/259] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:11 2022] PUT /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/acb28d4482ba46b99bc10057d74786d3/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:38:11.651700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f03b9-7c4b-42f5-8294-3dea979f7de5 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:11.651994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f03b9-7c4b-42f5-8294-3dea979f7de5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:11.652272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f03b9-7c4b-42f5-8294-3dea979f7de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:11.689925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7f7f03b9-7c4b-42f5-8294-3dea979f7de5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:11.691127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7f7f03b9-7c4b-42f5-8294-3dea979f7de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:11.724601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 132/260] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 12:38:11.728986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:11.833962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 129/261] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:11.869790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f03b9-7c4b-42f5-8294-3dea979f7de5 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:11.896517 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:11.914950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:11.948489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3b7f2-e601-4619-9b6a-dd25c00a8080 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:12.003879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:12.005042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:12.005326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:12.005598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:12.007024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:12.009088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:12.016514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 133/262] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:11 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:12.018186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:12.105541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f03b9-7c4b-42f5-8294-3dea979f7de5 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:12.132820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159e34c-889a-478a-9264-7e94a770b6f5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:12.171821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fc71c-3509-4166-b119-994d967490ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:12.172486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc71c-3509-4166-b119-994d967490ff None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:12.172766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc71c-3509-4166-b119-994d967490ff None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:12.173038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc71c-3509-4166-b119-994d967490ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:12.174259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fc71c-3509-4166-b119-994d967490ff None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=0bca04d6a1cb4f8d9bce3519d1e55515) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:12.174572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fc71c-3509-4166-b119-994d967490ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:12.176654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fc71c-3509-4166-b119-994d967490ff None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:12.187159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 134/263] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:12 2022] GET /identity/v3/role_assignments?user.id=0bca04d6a1cb4f8d9bce3519d1e55515&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:12.195980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:12.197216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:12.197718 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:12.198211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:12.224593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:12.233851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:12.234808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:12.235621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:12.238124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 130/264] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:12.244464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fc71c-3509-4166-b119-994d967490ff None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:12.263782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:12.369839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fc71c-3509-4166-b119-994d967490ff None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:12.410075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:12.410774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:12.411094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:12.411385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:12.412167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:12.412468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:12.414355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:12.423985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 135/265] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:12.429163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:12.547625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:12.558465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781ca93-331e-4b7f-a2a4-710104d4e4a8 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:12.595301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:12.596229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:12.596556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:12.596833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] PATH_INFO: `/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:12.598060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] RBAC: Authorizing `identity:get_user(user_id=0bca04d6a1cb4f8d9bce3519d1e55515)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:12.599549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:12.601241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 131/266] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:12 2022] GET /identity/v3/users/0bca04d6a1cb4f8d9bce3519d1e55515 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:12.606823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:12.681864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:12.719545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ec79-c179-4b6a-8f1e-8b4fa8eb5244 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:12.755017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363012cf-857a-49e0-a5aa-018e1e80036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:12.755733 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363012cf-857a-49e0-a5aa-018e1e80036b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:12.756019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363012cf-857a-49e0-a5aa-018e1e80036b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:12.756297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363012cf-857a-49e0-a5aa-018e1e80036b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:12.837033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363012cf-857a-49e0-a5aa-018e1e80036b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:12.838556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363012cf-857a-49e0-a5aa-018e1e80036b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:12.907471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:12.939929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 136/267] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:12.945694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.076478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c481-45c4-4674-b15b-b7ab78320e57 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:13.091111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40e281-a52c-48a9-9683-95fd286a1dff None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.091400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40e281-a52c-48a9-9683-95fd286a1dff None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.091618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40e281-a52c-48a9-9683-95fd286a1dff None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.093271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 137/268] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:13.098087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b0672-ca31-413a-98e3-012bacb490d7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.098377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b0672-ca31-413a-98e3-012bacb490d7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.098657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b0672-ca31-413a-98e3-012bacb490d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.117762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:13.119688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.119688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.120136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.121440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:13.125511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:13.125511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 132/269] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:12 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:13.131209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.151067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c74b0672-ca31-413a-98e3-012bacb490d7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:13.156322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c74b0672-ca31-413a-98e3-012bacb490d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:13.196996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:13.263853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 138/270] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:13.265653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af16d3b-f405-4106-88a3-846d4be111e8 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.266120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af16d3b-f405-4106-88a3-846d4be111e8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.266358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af16d3b-f405-4106-88a3-846d4be111e8 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.268007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 139/271] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:13.276643 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271c7ec-b87f-4acb-b54c-21d9e7ecf59c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.276967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271c7ec-b87f-4acb-b54c-21d9e7ecf59c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.277214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271c7ec-b87f-4acb-b54c-21d9e7ecf59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.291808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e942a26-88fc-443f-98da-9d2e405f60a0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:13.307244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a271c7ec-b87f-4acb-b54c-21d9e7ecf59c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:13.308594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a271c7ec-b87f-4acb-b54c-21d9e7ecf59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:13.335008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:13.336309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.337045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.337914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.339223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] RBAC: Adding query filter params (role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=0bca04d6a1cb4f8d9bce3519d1e55515) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:13.340076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:13.342413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:13.350215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 133/272] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:13 2022] GET /identity/v3/role_assignments?user.id=0bca04d6a1cb4f8d9bce3519d1e55515&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=20865fed4c894cccb7f1c223e95353ec => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:13.351899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.387830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 140/273] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:13.389427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c09e2-7608-409f-befd-8ecb3052fe94 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.389708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c09e2-7608-409f-befd-8ecb3052fe94 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.389988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c09e2-7608-409f-befd-8ecb3052fe94 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.391576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 141/274] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:13.397210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8181e2-1e7b-4999-89f9-eb4c14e83a54 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.397555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8181e2-1e7b-4999-89f9-eb4c14e83a54 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.397838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8181e2-1e7b-4999-89f9-eb4c14e83a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.431269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-da8181e2-1e7b-4999-89f9-eb4c14e83a54 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:13.432446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-da8181e2-1e7b-4999-89f9-eb4c14e83a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:13.464190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:13.496778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:13.501710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f5f2e-ed50-4543-b410-ac22d7ad80e9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:13.519675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 142/275] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:13.521098 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8181e2-1e7b-4999-89f9-eb4c14e83a54 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.547713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:13.548542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.548828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.549108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.562148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:13.569958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:13.570905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:13.571722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 134/276] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:13.572435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.662517 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8181e2-1e7b-4999-89f9-eb4c14e83a54 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:13.697764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:13.698719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.699006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.699337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.700006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d66458-8e1c-4e03-b02c-46641edeb449 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:13.700674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:13.702141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:13.708399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 143/277] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:13.708399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.736723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:13.737545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.737831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.738110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.738920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:13.751024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:13.758565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:13.759457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:13.760234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 135/278] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:13.760900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.776254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:13.820513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc47541-0189-4efa-b898-033b395d04f2 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:13.856399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:13.857073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.857357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.857912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.858384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] RBAC: Adding query filter params (name=reader) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:13.858691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:13.860109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:13.869144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 144/279] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:13.869769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.874620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f10fc-9cab-43e7-8e18-81e806c78f3c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:13.910280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:13.910978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:13.911262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:13.911571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:13.912292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:13.914281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:13.947403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 136/280] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:13 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:13.947403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:13.980465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:13.992418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc558-e4c9-4566-90cc-a43b9cbac773 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:14.021262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:14.044816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:14.045520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.045817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.046086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.046872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:14.047175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:14.048615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:14.063592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 145/281] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:14.067424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:14.078482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240231-8186-424c-ab57-f37c4cd70b9b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:14.123975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:14.124880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.125166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.125446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] PATH_INFO: `/v3/users/4831118709fe4e5db8d82bce71fdd83c` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.126835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] RBAC: Authorizing `identity:get_user(user_id=4831118709fe4e5db8d82bce71fdd83c)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:14.128384 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:14.130114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 137/282] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:13 2022] GET /identity/v3/users/4831118709fe4e5db8d82bce71fdd83c => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:14.136571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:14.211982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175af1a-2d67-4919-b8d7-def58c2c8630 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:14.257651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:14.258992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.259537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.259914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] PATH_INFO: `/v3/users/7dc49250d810460fb5b8779b87672fa4` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.279177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] RBAC: Authorizing `identity:get_user(user_id=7dc49250d810460fb5b8779b87672fa4)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:14.279838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1651081-3dd7-49f3-bf12-c03889e8a080 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:14.280896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:14.283129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 146/283] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:14 2022] GET /identity/v3/users/7dc49250d810460fb5b8779b87672fa4 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:14.289553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:14.310954 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:14.328287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:14.329448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.329738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.330018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.331570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:14.335127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:14.339472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 138/284] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:14 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:14.343481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:14.347660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:14.423665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710feef-6039-4f74-9720-b0b395e20bd2 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:14.458400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d73d89-0fa3-4642-b48e-94f118557351 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:14.462688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efdcac-84ac-4562-928a-37893c768c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:14.463709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efdcac-84ac-4562-928a-37893c768c1b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.464274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efdcac-84ac-4562-928a-37893c768c1b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.464748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efdcac-84ac-4562-928a-37893c768c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.465902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efdcac-84ac-4562-928a-37893c768c1b None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, user.id=7dc49250d810460fb5b8779b87672fa4, scope.system=all) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:14.466415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efdcac-84ac-4562-928a-37893c768c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:14.468221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efdcac-84ac-4562-928a-37893c768c1b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:14.474669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 147/285] 10.222.0.87 () {56 vars in 1308 bytes} [Fri Jun 17 12:38:14 2022] GET /identity/v3/role_assignments?user.id=7dc49250d810460fb5b8779b87672fa4&scope.system=all&role.id=fed756b9361747358b9a5948eb9a1442 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:14.497494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6264b4-809c-447a-809d-d7973812f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:14.498451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6264b4-809c-447a-809d-d7973812f8c0 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.498734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6264b4-809c-447a-809d-d7973812f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.499017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6264b4-809c-447a-809d-d7973812f8c0 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/4831118709fe4e5db8d82bce71fdd83c/roles/2f177b995bd14bb4b85362a4f62b975d` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.501356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6264b4-809c-447a-809d-d7973812f8c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd408398a4fd4a1d970960d255e9e000, user_id=4831118709fe4e5db8d82bce71fdd83c, role_id=2f177b995bd14bb4b85362a4f62b975d)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:14.502889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6264b4-809c-447a-809d-d7973812f8c0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:14.519543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 139/286] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:14 2022] PUT /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000/users/4831118709fe4e5db8d82bce71fdd83c/roles/2f177b995bd14bb4b85362a4f62b975d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:38:14.560568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42210a-36db-4907-a276-7082b9f7a4ab None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.561256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42210a-36db-4907-a276-7082b9f7a4ab None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.561779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42210a-36db-4907-a276-7082b9f7a4ab None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.563542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 148/287] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:14.569510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e725fb-0007-42bf-9351-b7e8da314207 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.569866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e725fb-0007-42bf-9351-b7e8da314207 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.570088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e725fb-0007-42bf-9351-b7e8da314207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.595982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-74e725fb-0007-42bf-9351-b7e8da314207 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:14.597300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-74e725fb-0007-42bf-9351-b7e8da314207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:14.706642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:14.714572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 140/288] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:14.740232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42210a-36db-4907-a276-7082b9f7a4ab None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:14.842876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:14.871574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42210a-36db-4907-a276-7082b9f7a4ab None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:14.918802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:14.920042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:14.920556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:14.921029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:14.932363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:14.940737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:14.941631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:14.942430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:14.947844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 149/289] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:14.951291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e725fb-0007-42bf-9351-b7e8da314207 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:15.154324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e725fb-0007-42bf-9351-b7e8da314207 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:15.204006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:15.204686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:15.204969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:15.205241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:15.206000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] RBAC: Adding query filter params (name=member) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:15.206306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:15.207716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:15.223006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 141/290] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:15.229138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:15.359834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940b4db-51b0-423e-ba5a-aa593e4965f4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:15.408214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:15.409040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:15.409322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:15.409592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] PATH_INFO: `/v3/users/acb28d4482ba46b99bc10057d74786d3` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:15.410860 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] RBAC: Authorizing `identity:get_user(user_id=acb28d4482ba46b99bc10057d74786d3)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:15.412356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:15.414818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 150/291] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:15 2022] GET /identity/v3/users/acb28d4482ba46b99bc10057d74786d3 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:15.420510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:15.506814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:15.558927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd58dd2-74ab-4b38-a014-7c30604b7871 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:15.609901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:15.610563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:15.610848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:15.611125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:15.706786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:15.708196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:15.788015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:15.855159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 142/292] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:15.860863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:15.883466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:16.126245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23130ad-f342-40a7-8fc0-ade63f5072d1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:16.185095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c72fc4-57ae-40bb-8a40-d181c9bd5ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:16.185969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72fc4-57ae-40bb-8a40-d181c9bd5ce9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:16.186254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72fc4-57ae-40bb-8a40-d181c9bd5ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:16.186532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72fc4-57ae-40bb-8a40-d181c9bd5ce9 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:16.192086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c72fc4-57ae-40bb-8a40-d181c9bd5ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:16.193561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c72fc4-57ae-40bb-8a40-d181c9bd5ce9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:16.195363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 151/293] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:15 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:16.201525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:16.336691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7b17-65f1-4b7a-bd74-e0ff9bb14c07 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:16.390062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9817bf0-6b97-4676-9af6-9e2f4f413383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:16.390657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9817bf0-6b97-4676-9af6-9e2f4f413383 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:16.390938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9817bf0-6b97-4676-9af6-9e2f4f413383 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:16.391210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9817bf0-6b97-4676-9af6-9e2f4f413383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:16.392093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9817bf0-6b97-4676-9af6-9e2f4f413383 None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=acb28d4482ba46b99bc10057d74786d3) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:16.392407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9817bf0-6b97-4676-9af6-9e2f4f413383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:16.393880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9817bf0-6b97-4676-9af6-9e2f4f413383 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:16.405633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 143/294] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:16 2022] GET /identity/v3/role_assignments?user.id=acb28d4482ba46b99bc10057d74786d3&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=fed756b9361747358b9a5948eb9a1442 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:16.668332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234641-331b-415e-bee5-cd9f5ebcfddb None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:16.668825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234641-331b-415e-bee5-cd9f5ebcfddb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:16.668825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234641-331b-415e-bee5-cd9f5ebcfddb None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:16.670334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 152/295] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:16.683153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560df5-923c-4ed0-93d1-48d50e1392ef None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:16.683480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560df5-923c-4ed0-93d1-48d50e1392ef None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:16.683751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560df5-923c-4ed0-93d1-48d50e1392ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:16.703975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-5a560df5-923c-4ed0-93d1-48d50e1392ef None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:16.705193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5a560df5-923c-4ed0-93d1-48d50e1392ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:16.721013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:16.786391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 144/296] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:16.811346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26234641-331b-415e-bee5-cd9f5ebcfddb None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:16.891016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:16.910272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e83ac-ffc1-461a-8e61-afdb8b5d3d8c None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:16.910563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e83ac-ffc1-461a-8e61-afdb8b5d3d8c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:16.910840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e83ac-ffc1-461a-8e61-afdb8b5d3d8c None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:16.912666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 145/297] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:16.918972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3a879-5f23-4ccd-90eb-70f78007d1bb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:16.919258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3a879-5f23-4ccd-90eb-70f78007d1bb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:16.919487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3a879-5f23-4ccd-90eb-70f78007d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:16.937608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7dd3a879-5f23-4ccd-90eb-70f78007d1bb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:16.938858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7dd3a879-5f23-4ccd-90eb-70f78007d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:16.944970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26234641-331b-415e-bee5-cd9f5ebcfddb None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:16.995174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d936e-3a1e-407e-b66b-5951fcd04ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:16.996943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d936e-3a1e-407e-b66b-5951fcd04ecb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:16.997688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d936e-3a1e-407e-b66b-5951fcd04ecb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:16.998200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d936e-3a1e-407e-b66b-5951fcd04ecb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.009674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d936e-3a1e-407e-b66b-5951fcd04ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:17.017764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 146/298] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-159d936e-3a1e-407e-b66b-5951fcd04ecb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:17.017902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:17.018717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:17.019511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 153/299] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:17.020172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58f62-eac2-4bf1-aed3-e3d6f45fad9d None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.020172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58f62-eac2-4bf1-aed3-e3d6f45fad9d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.020172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58f62-eac2-4bf1-aed3-e3d6f45fad9d None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.021303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 147/300] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:17.028804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5ddef-fa37-4736-9985-7ed0d5fbc462 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.029094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5ddef-fa37-4736-9985-7ed0d5fbc462 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.029376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5ddef-fa37-4736-9985-7ed0d5fbc462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.032929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58f62-eac2-4bf1-aed3-e3d6f45fad9d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.049141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-83e5ddef-fa37-4736-9985-7ed0d5fbc462 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:17.050346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-83e5ddef-fa37-4736-9985-7ed0d5fbc462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:17.126043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 154/301] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:17.126905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5ddef-fa37-4736-9985-7ed0d5fbc462 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.138092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.165187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58f62-eac2-4bf1-aed3-e3d6f45fad9d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.191196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.203168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.203873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.204153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.204426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.205176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] RBAC: Adding query filter params (name=reader) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:17.205478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.206837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:17.216480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 148/302] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:17.217201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.236905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5ddef-fa37-4736-9985-7ed0d5fbc462 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.275768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.276583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.277078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.277184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.278301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.279871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:17.285927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 155/303] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:17.285927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.322862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f775741-38e4-4561-b620-e57c8a656110 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.357815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.358687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.358972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.359273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.369854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:17.377851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:17.378895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 149/304] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:17.385068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.389148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.404225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2f50-1118-4131-822d-dfdd62cf7b4b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.436651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.443991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.444812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.445103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.445564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] PATH_INFO: `/v3/users/190ffdd855f74fd4a38c4c2131a25e68` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.461171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] RBAC: Authorizing `identity:get_user(user_id=190ffdd855f74fd4a38c4c2131a25e68)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.463340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:17.465191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 156/305] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/users/190ffdd855f74fd4a38c4c2131a25e68 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:17.465998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.499328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e975e08-84c7-498e-acba-827186a6ea17 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.532915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.533591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.533875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.534151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] PATH_INFO: `/v3/groups` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.534845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.536341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:17.550939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 150/306] 10.222.0.87 () {60 vars in 1134 bytes} [Fri Jun 17 12:38:17 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:17.551767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.575525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d9f7e-7597-444c-a7c2-ef750f43f86a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.614563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c975b8-cb61-431b-8d1c-d105cb5af9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.615243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c975b8-cb61-431b-8d1c-d105cb5af9b7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.615580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c975b8-cb61-431b-8d1c-d105cb5af9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.615795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c975b8-cb61-431b-8d1c-d105cb5af9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.616599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c975b8-cb61-431b-8d1c-d105cb5af9b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:17.616905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c975b8-cb61-431b-8d1c-d105cb5af9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.618886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c975b8-cb61-431b-8d1c-d105cb5af9b7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:17.625982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.628494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 157/307] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:17.629883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49188c-0b6d-4388-aa2a-03d6bd2c68e3 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.630174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49188c-0b6d-4388-aa2a-03d6bd2c68e3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.630454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49188c-0b6d-4388-aa2a-03d6bd2c68e3 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.635635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 158/308] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:17.635635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49188c-0b6d-4388-aa2a-03d6bd2c68e3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.680629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b7150-4736-4198-8f46-0eb48bf07c53 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.698060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.721407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2d8fe-dac0-4e41-979b-dfdbd1eccecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.722093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d8fe-dac0-4e41-979b-dfdbd1eccecc None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.722435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d8fe-dac0-4e41-979b-dfdbd1eccecc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.723398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d8fe-dac0-4e41-979b-dfdbd1eccecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.723617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2d8fe-dac0-4e41-979b-dfdbd1eccecc None admin] RBAC: Adding query filter params (role.id=2f177b995bd14bb4b85362a4f62b975d, user.id=190ffdd855f74fd4a38c4c2131a25e68, scope.system=all) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:17.723934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2d8fe-dac0-4e41-979b-dfdbd1eccecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.725394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2d8fe-dac0-4e41-979b-dfdbd1eccecc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:17.732342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 151/309] 10.222.0.87 () {56 vars in 1308 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/role_assignments?user.id=190ffdd855f74fd4a38c4c2131a25e68&scope.system=all&role.id=2f177b995bd14bb4b85362a4f62b975d => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:17.732552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106373c5-7750-490e-a873-4c0b7ece29ef None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.732961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106373c5-7750-490e-a873-4c0b7ece29ef None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.733136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106373c5-7750-490e-a873-4c0b7ece29ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.752759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-106373c5-7750-490e-a873-4c0b7ece29ef None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:17.754040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-106373c5-7750-490e-a873-4c0b7ece29ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:17.765660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49188c-0b6d-4388-aa2a-03d6bd2c68e3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.815958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.817126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.817601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.817884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] PATH_INFO: `/v3/users/7dc49250d810460fb5b8779b87672fa4` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.819171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] RBAC: Authorizing `identity:get_user(user_id=7dc49250d810460fb5b8779b87672fa4)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.820875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:17.822862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 159/310] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/users/7dc49250d810460fb5b8779b87672fa4 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:17.828580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.843895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 152/311] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:17.867358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106373c5-7750-490e-a873-4c0b7ece29ef None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:17.930801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.951757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541f4a4-9a56-4906-8926-84fe0bbb7e60 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.970205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:17.973339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106373c5-7750-490e-a873-4c0b7ece29ef None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:17.989655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:17.990877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:17.991304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:17.991792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] PATH_INFO: `/v3/system/users/7dc49250d810460fb5b8779b87672fa4/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:17.993558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7dc49250d810460fb5b8779b87672fa4, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:17.995217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:18.007811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 160/312] 10.222.0.87 () {58 vars in 1334 bytes} [Fri Jun 17 12:38:17 2022] PUT /identity/v3/system/users/7dc49250d810460fb5b8779b87672fa4/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:38:18.009948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:18.010986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:18.011272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:18.011580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:18.030155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:18.039203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:18.040364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:18.041144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 153/313] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:18.047410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:18.171165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f38e6-ea57-441c-a4a4-fe16f1e962ba None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:18.207962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:18.208566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:18.208848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:18.209129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:18.209890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:18.210199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:18.211746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:18.220751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 161/314] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:18.226230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:18.328349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c61ee-8075-4de7-b7da-c5a028fd9f15 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:18.349113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:18.367488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:18.368498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:18.369000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:18.369277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] PATH_INFO: `/v3/users/4831118709fe4e5db8d82bce71fdd83c` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:18.370504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] RBAC: Authorizing `identity:get_user(user_id=4831118709fe4e5db8d82bce71fdd83c)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:18.371998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:18.374033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 154/315] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:18 2022] GET /identity/v3/users/4831118709fe4e5db8d82bce71fdd83c => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:18.379327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:18.427224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:18.484824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a7fbd-640c-4293-abc3-09ee36fc96ae None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:18.520767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:18.521818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:18.522302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:18.522872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:18.595200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:18.596739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:18.641496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:18.701888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 162/316] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:18.707304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:18.814891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a153988-762f-486b-8438-0f4020e1fc76 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:18.849586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60368700-f67a-45d0-be63-c3595e1763b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:18.850472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368700-f67a-45d0-be63-c3595e1763b5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:18.850763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368700-f67a-45d0-be63-c3595e1763b5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:18.851170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368700-f67a-45d0-be63-c3595e1763b5 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:18.852430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60368700-f67a-45d0-be63-c3595e1763b5 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:18.853887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60368700-f67a-45d0-be63-c3595e1763b5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:18.855702 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 155/317] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:18 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:18.861836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:18.966243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59a76-93c6-4bdf-a0de-dc1884513ff6 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:19.001972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ebd41-0469-4d05-b4e0-7c673f99f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:19.002648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ebd41-0469-4d05-b4e0-7c673f99f079 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:19.002935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ebd41-0469-4d05-b4e0-7c673f99f079 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:19.003215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ebd41-0469-4d05-b4e0-7c673f99f079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:19.004096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ebd41-0469-4d05-b4e0-7c673f99f079 None admin] RBAC: Adding query filter params (role.id=2f177b995bd14bb4b85362a4f62b975d, scope.project.id=cd408398a4fd4a1d970960d255e9e000, user.id=4831118709fe4e5db8d82bce71fdd83c) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:19.004410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ebd41-0469-4d05-b4e0-7c673f99f079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:19.005877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ebd41-0469-4d05-b4e0-7c673f99f079 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:19.012166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 163/318] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:18 2022] GET /identity/v3/role_assignments?user.id=4831118709fe4e5db8d82bce71fdd83c&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=2f177b995bd14bb4b85362a4f62b975d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:19.872119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f7536-0e7e-42cc-b3f1-4d4339b51965 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:19.873104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f7536-0e7e-42cc-b3f1-4d4339b51965 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:19.873662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f7536-0e7e-42cc-b3f1-4d4339b51965 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:19.875496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 156/319] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:19.880898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990d347-91eb-444b-9be7-b8d63b7fdf75 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:19.881330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990d347-91eb-444b-9be7-b8d63b7fdf75 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:19.881524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990d347-91eb-444b-9be7-b8d63b7fdf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:19.901534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e990d347-91eb-444b-9be7-b8d63b7fdf75 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:19.902796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e990d347-91eb-444b-9be7-b8d63b7fdf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:19.905537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:19.968443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0aacc-affc-46ff-8b40-4718091991a9 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:19.968761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0aacc-affc-46ff-8b40-4718091991a9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:19.968998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0aacc-affc-46ff-8b40-4718091991a9 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:19.970377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 157/320] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:19.975207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67f14c-6940-422d-a0c4-e03779c6a2ef None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:19.975517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67f14c-6940-422d-a0c4-e03779c6a2ef None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:19.975815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67f14c-6940-422d-a0c4-e03779c6a2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:19.979724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 164/321] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:19.992488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-df67f14c-6940-422d-a0c4-e03779c6a2ef None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:19.993857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-df67f14c-6940-422d-a0c4-e03779c6a2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:20.002761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990d347-91eb-444b-9be7-b8d63b7fdf75 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.043328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:20.064775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 158/322] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:20.087331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67f14c-6940-422d-a0c4-e03779c6a2ef None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.116475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990d347-91eb-444b-9be7-b8d63b7fdf75 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.153573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.154289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.154572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.154858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.156071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:20.157584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:20.159281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 165/323] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:20.166461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.176530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:20.207932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67f14c-6940-422d-a0c4-e03779c6a2ef None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.249609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459c7a4-3f40-4bb9-987e-2c3a3079feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.250431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459c7a4-3f40-4bb9-987e-2c3a3079feea None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.250717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459c7a4-3f40-4bb9-987e-2c3a3079feea None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.250994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459c7a4-3f40-4bb9-987e-2c3a3079feea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.261943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459c7a4-3f40-4bb9-987e-2c3a3079feea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-7459c7a4-3f40-4bb9-987e-2c3a3079feea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:20.269574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:20.270442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:20.271459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 159/324] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:20.272286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd918-9b86-4d28-9a87-121e62971021 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.272601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd918-9b86-4d28-9a87-121e62971021 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.273346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd918-9b86-4d28-9a87-121e62971021 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.274667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 160/325] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:20.277397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd918-9b86-4d28-9a87-121e62971021 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.285223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79d983-7455-4d11-9a89-47c6cd7bb3da None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.318143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf70413-d8a9-452c-9019-2ae68b77e500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.318816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf70413-d8a9-452c-9019-2ae68b77e500 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.319104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf70413-d8a9-452c-9019-2ae68b77e500 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.319403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf70413-d8a9-452c-9019-2ae68b77e500 None admin] PATH_INFO: `/v3/groups` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.320116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf70413-d8a9-452c-9019-2ae68b77e500 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:20.320821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:20.321609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf70413-d8a9-452c-9019-2ae68b77e500 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:20.337349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 166/326] 10.222.0.87 () {60 vars in 1134 bytes} [Fri Jun 17 12:38:20 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:38:20.338668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bcf86-ceb4-4000-8fe5-b3abcdd115c1 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.338961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bcf86-ceb4-4000-8fe5-b3abcdd115c1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.339397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bcf86-ceb4-4000-8fe5-b3abcdd115c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.365931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-639bcf86-ceb4-4000-8fe5-b3abcdd115c1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:20.367183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-639bcf86-ceb4-4000-8fe5-b3abcdd115c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:20.393832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd918-9b86-4d28-9a87-121e62971021 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.432409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.433119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.433405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.433803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.434578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:20.434893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:20.436455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:20.437315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:20.446575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 167/327] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:20.449823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 161/328] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:20.457150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bcf86-ceb4-4000-8fe5-b3abcdd115c1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.470059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.575128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bcf86-ceb4-4000-8fe5-b3abcdd115c1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.588518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67313f8-08dc-4d11-bb64-3a47d1459a20 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.591693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:20.622988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.624963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.624963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.624963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] PATH_INFO: `/v3/users/190ffdd855f74fd4a38c4c2131a25e68` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.626450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=190ffdd855f74fd4a38c4c2131a25e68)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:20.628388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:20.630666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 168/329] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:20 2022] GET /identity/v3/users/190ffdd855f74fd4a38c4c2131a25e68 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:20.635600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.636420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.636704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.636806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.636980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.649889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:20.657708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:20.658711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:20.659714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 162/330] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:20.666397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.713129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:20.759136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c233f30-abaf-4066-825c-c9743cd49fc4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.797369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe489a-8eea-4262-8636-eba1f607753a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:20.800499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.801551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.801971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.802408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] PATH_INFO: `/v3/system/users/190ffdd855f74fd4a38c4c2131a25e68/roles/2f177b995bd14bb4b85362a4f62b975d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.805198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=190ffdd855f74fd4a38c4c2131a25e68, role_id=2f177b995bd14bb4b85362a4f62b975d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:20.806610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:20.818307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 169/331] 10.222.0.87 () {58 vars in 1334 bytes} [Fri Jun 17 12:38:20 2022] PUT /identity/v3/system/users/190ffdd855f74fd4a38c4c2131a25e68/roles/2f177b995bd14bb4b85362a4f62b975d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:20.837958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:20.838639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:20.838924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:20.839199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:20.840069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] RBAC: Adding query filter params (name=member) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:20.840379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:20.841803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:20.852814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 163/332] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:20.858635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:20.996823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809478db-7423-40ca-bc19-61e986d10b58 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:21.026900 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:21.036407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8be3c9-54f0-49c1-a2a0-c84355593238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:21.037335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be3c9-54f0-49c1-a2a0-c84355593238 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:21.038454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be3c9-54f0-49c1-a2a0-c84355593238 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:21.038732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be3c9-54f0-49c1-a2a0-c84355593238 None admin] PATH_INFO: `/v3/users/7dc49250d810460fb5b8779b87672fa4` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:21.040886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8be3c9-54f0-49c1-a2a0-c84355593238 None admin] RBAC: Authorizing `identity:get_user(user_id=7dc49250d810460fb5b8779b87672fa4)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:21.042446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8be3c9-54f0-49c1-a2a0-c84355593238 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:21.044917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 170/333] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:20 2022] GET /identity/v3/users/7dc49250d810460fb5b8779b87672fa4 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:21.050709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:21.107759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:21.173500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b979a-b18b-480a-bde0-d398f99a5012 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:21.214736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a040900-a071-4a5a-8d77-db2cfacf87f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:21.215445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a040900-a071-4a5a-8d77-db2cfacf87f3 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:21.215732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a040900-a071-4a5a-8d77-db2cfacf87f3 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:21.216009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a040900-a071-4a5a-8d77-db2cfacf87f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:21.216866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a040900-a071-4a5a-8d77-db2cfacf87f3 None admin] RBAC: Adding query filter params (role.id=fed756b9361747358b9a5948eb9a1442, user.id=7dc49250d810460fb5b8779b87672fa4, scope.system=all) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:21.217178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a040900-a071-4a5a-8d77-db2cfacf87f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:21.218618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a040900-a071-4a5a-8d77-db2cfacf87f3 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:21.233932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 164/334] 10.222.0.87 () {56 vars in 1308 bytes} [Fri Jun 17 12:38:21 2022] GET /identity/v3/role_assignments?user.id=7dc49250d810460fb5b8779b87672fa4&scope.system=all&role.id=fed756b9361747358b9a5948eb9a1442 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:23.234225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c42c3-58e5-4e5f-9fe8-f00a71cf3280 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.234800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c42c3-58e5-4e5f-9fe8-f00a71cf3280 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.234800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c42c3-58e5-4e5f-9fe8-f00a71cf3280 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.236172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 171/335] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:23.252277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac729-112b-4296-96e6-d8266fb64d8b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.252812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac729-112b-4296-96e6-d8266fb64d8b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.252812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac729-112b-4296-96e6-d8266fb64d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.275068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a55ac729-112b-4296-96e6-d8266fb64d8b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:23.276299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a55ac729-112b-4296-96e6-d8266fb64d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:23.369243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 165/336] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:23.399717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c42c3-58e5-4e5f-9fe8-f00a71cf3280 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:23.494818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36911f18-6860-44a7-bdce-28ddf3b5fc6f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.495175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36911f18-6860-44a7-bdce-28ddf3b5fc6f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.495390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36911f18-6860-44a7-bdce-28ddf3b5fc6f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.496971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 166/337] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:23.509130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db53d-8e10-47e3-91fd-b71ba4a89972 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.509513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db53d-8e10-47e3-91fd-b71ba4a89972 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.509754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db53d-8e10-47e3-91fd-b71ba4a89972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.529584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c42c3-58e5-4e5f-9fe8-f00a71cf3280 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:23.554623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c19db53d-8e10-47e3-91fd-b71ba4a89972 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:23.556251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c19db53d-8e10-47e3-91fd-b71ba4a89972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:23.569870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8addd98-68f8-4a25-99b4-25d345a20613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:23.571198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addd98-68f8-4a25-99b4-25d345a20613 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.572035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addd98-68f8-4a25-99b4-25d345a20613 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.572315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addd98-68f8-4a25-99b4-25d345a20613 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.579721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:23.583465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8addd98-68f8-4a25-99b4-25d345a20613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-c8addd98-68f8-4a25-99b4-25d345a20613 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:23.597640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:23.598594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:23.599428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:23.600059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 172/338] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:23.600892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fad169-e08d-4e64-a27d-5c4a2ed14bb9 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.601176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fad169-e08d-4e64-a27d-5c4a2ed14bb9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.601449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fad169-e08d-4e64-a27d-5c4a2ed14bb9 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.609903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 173/339] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:23.610682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fad169-e08d-4e64-a27d-5c4a2ed14bb9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:23.710227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 167/340] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:23.711567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf703-c52f-4ccd-a8bf-7bd9b139b4f6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.712062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf703-c52f-4ccd-a8bf-7bd9b139b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.712403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf703-c52f-4ccd-a8bf-7bd9b139b4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.731734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:23.748737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8e7bf703-c52f-4ccd-a8bf-7bd9b139b4f6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:23.749923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8e7bf703-c52f-4ccd-a8bf-7bd9b139b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:23.830354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fad169-e08d-4e64-a27d-5c4a2ed14bb9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:23.876090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:23.876808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:23.877179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:23.877458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:23.878722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:23.879168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:23.880735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:23.893663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 174/341] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:23.894623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:23.898404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 168/342] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:23.899457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bf703-c52f-4ccd-a8bf-7bd9b139b4f6 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:24.019735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006068ed-9a13-41b9-84ec-adfc2ef71f02 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.020414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.041309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bf703-c52f-4ccd-a8bf-7bd9b139b4f6 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.042672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.083745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.085125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.085455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.085734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.089533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.090687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.091042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.091438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.099496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:24.109936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:24.110863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:24.111738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 175/343] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:24.112404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:24.197100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:24.198576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:24.247259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cadf8b-6b88-4583-9e33-c3e406fadc9f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.300308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.303955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328dce8-c4bb-4418-935f-82f694a9267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.305074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328dce8-c4bb-4418-935f-82f694a9267e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.305574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328dce8-c4bb-4418-935f-82f694a9267e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.306688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328dce8-c4bb-4418-935f-82f694a9267e None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.318809 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328dce8-c4bb-4418-935f-82f694a9267e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-c328dce8-c4bb-4418-935f-82f694a9267e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:24.328470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:24.329408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:24.330215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:24.332762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 176/344] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:24.338124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 169/345] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:24.338124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:24.339161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a750f07-6b1a-490f-8fd8-b5eda7ce6db9 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.339667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a750f07-6b1a-490f-8fd8-b5eda7ce6db9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.340149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a750f07-6b1a-490f-8fd8-b5eda7ce6db9 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.341724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 177/346] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:24.342614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a750f07-6b1a-490f-8fd8-b5eda7ce6db9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:24.393633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.466926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a750f07-6b1a-490f-8fd8-b5eda7ce6db9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.473544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b7597-59a9-4576-9f5a-be43089aff19 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.505152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.506197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.506694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.507162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.508275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:24.509057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:24.510740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:24.517588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fba4a-e8be-4a37-a7ef-8f68a1399729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.518273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fba4a-e8be-4a37-a7ef-8f68a1399729 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.518562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fba4a-e8be-4a37-a7ef-8f68a1399729 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.518840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fba4a-e8be-4a37-a7ef-8f68a1399729 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.519749 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fba4a-e8be-4a37-a7ef-8f68a1399729 None admin] RBAC: Adding query filter params (name=member) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:24.521834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fba4a-e8be-4a37-a7ef-8f68a1399729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:24.524089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 178/347] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:24.524487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fba4a-e8be-4a37-a7ef-8f68a1399729 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:24.527065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:24.542832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 170/348] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:24.545912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a697-8a6b-4405-ba36-781e616ffc25 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.545912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a697-8a6b-4405-ba36-781e616ffc25 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.545912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a697-8a6b-4405-ba36-781e616ffc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.568616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-2295a697-8a6b-4405-ba36-781e616ffc25 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:24.570346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2295a697-8a6b-4405-ba36-781e616ffc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:24.617988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.650280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.654652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626853da-8361-4327-bd97-d1eb594b7f43 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.674119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 171/349] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:24.675147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295a697-8a6b-4405-ba36-781e616ffc25 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:24.702286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.703646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.704129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.704597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.706086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:24.707829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:24.710104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 179/350] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:24 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:24.711084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:24.807254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295a697-8a6b-4405-ba36-781e616ffc25 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.849158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fceb546-f459-4412-99b4-8f3256e89c86 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:24.855998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.856666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.857002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.857285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.895403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:24.896861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:24.897144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:24.897605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:24.904380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.944706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:24.949560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:24.952355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:24.992694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:24.994785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:25.103166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 172/351] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:25.104103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.135828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 180/352] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:25.137260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.193733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:25.226958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:25.234434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724f89e-77fb-4beb-a702-63ec9bc8c53e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:25.268995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f74be-47cb-4807-8238-672cab37ca17 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:25.275059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df949886-feac-48dd-ad3f-bd2f4bf1a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:25.276011 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949886-feac-48dd-ad3f-bd2f4bf1a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.276296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949886-feac-48dd-ad3f-bd2f4bf1a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.277027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949886-feac-48dd-ad3f-bd2f4bf1a8c7 None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.284746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df949886-feac-48dd-ad3f-bd2f4bf1a8c7 None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:25.287271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df949886-feac-48dd-ad3f-bd2f4bf1a8c7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:25.289747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 173/353] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:25.291122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1b91-2250-4da7-ae4e-7c3fa0bc27b0 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.291469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1b91-2250-4da7-ae4e-7c3fa0bc27b0 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.291738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1b91-2250-4da7-ae4e-7c3fa0bc27b0 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.293110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 174/354] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:25.294033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb956477-35a4-463d-aef6-384d3ced845a None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.294324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb956477-35a4-463d-aef6-384d3ced845a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.294633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb956477-35a4-463d-aef6-384d3ced845a None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.295849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 175/355] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:25.296537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb956477-35a4-463d-aef6-384d3ced845a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.308518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:25.309343 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.309627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.309903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.321661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:25.330467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:25.331348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 12:38:25.332325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:25.335600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 181/356] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:25.336623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.425031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb956477-35a4-463d-aef6-384d3ced845a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:25.450061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fffec-d449-4dd7-b7d1-31969ece4046 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:25.458810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:25.467323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30320a8a-65dd-46c4-8319-70f710e97464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:25.468196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30320a8a-65dd-46c4-8319-70f710e97464 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.468475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30320a8a-65dd-46c4-8319-70f710e97464 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.470989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30320a8a-65dd-46c4-8319-70f710e97464 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.470989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30320a8a-65dd-46c4-8319-70f710e97464 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:25.472154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30320a8a-65dd-46c4-8319-70f710e97464 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:25.474123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 176/357] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:25.475473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d62f1-3a79-45f9-8c77-ac5ad9ce2b06 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.475473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d62f1-3a79-45f9-8c77-ac5ad9ce2b06 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.475653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d62f1-3a79-45f9-8c77-ac5ad9ce2b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.482224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:25.492765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:25.493666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.493951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.494233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.496019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:25.497505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:25.498084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-051d62f1-3a79-45f9-8c77-ac5ad9ce2b06 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:25.499231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 182/358] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:25.499343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-051d62f1-3a79-45f9-8c77-ac5ad9ce2b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:25.500097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.584698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 177/359] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:25.588738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df534b2-7af3-4554-939b-0deecfc207f4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.589142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df534b2-7af3-4554-939b-0deecfc207f4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.589427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df534b2-7af3-4554-939b-0deecfc207f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.607662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3df534b2-7af3-4554-939b-0deecfc207f4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:25.609130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3df534b2-7af3-4554-939b-0deecfc207f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:25.619837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efca5f-2169-418c-b9cb-4a6cc667dd31 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:25.660928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:25.661604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.661887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.662157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.662997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=3505cab8f3094ff785667548ec5311ca) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:25.663306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:25.666602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:25.672538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 183/360] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=20865fed4c894cccb7f1c223e95353ec => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:25.673347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.697297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 178/361] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:25.698603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df534b2-7af3-4554-939b-0deecfc207f4 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.779079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:25.782265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:25.828777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152126-ad35-4aef-8119-aa96fc7d519c None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:25.834618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df534b2-7af3-4554-939b-0deecfc207f4 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:25.871361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:25.871772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.872019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.872298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.873320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:25.873636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:25.875182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:25.880138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:25.881021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:25.881305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:25.881577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:25.882791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:25.884650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:25.885895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 184/362] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:25.886902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:25.891510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 179/363] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:25.892248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.014488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14c40-1005-441f-bf14-4078716cac49 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.014897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854820-aa5c-47dc-91f5-1374cca8ad79 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.048169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:26.053089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:26.065371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.066363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.066725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.067006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.071667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.073255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.073911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.074384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.076605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:26.078281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:26.080549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 185/364] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:26.081539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.086309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:26.094294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:26.095170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 180/365] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:26.096813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.214174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578860f-91e3-49b5-a3a0-6f407b5df7b7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.246517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42e09-5a81-4290-a060-c0df329d9fd4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.254184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.255190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.255656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.256094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.269662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:26.278708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:26.279623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:26.280404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 186/366] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:26.281047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.293259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.294161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.294442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.294719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] PATH_INFO: `/v3/users/190ffdd855f74fd4a38c4c2131a25e68` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.296079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] RBAC: Authorizing `identity:get_user(user_id=190ffdd855f74fd4a38c4c2131a25e68)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:26.297536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:26.303463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 181/367] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/users/190ffdd855f74fd4a38c4c2131a25e68 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:26.304312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.342760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:26.375738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:26.418673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cd199-bfee-4684-8fa9-67350495a31b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.439014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d0a50-5b3b-4efe-ba5c-79dc18f08537 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.471402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.472480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.472764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.473103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.474039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=cd408398a4fd4a1d970960d255e9e000) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:26.474506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:26.478460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:26.479879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.480736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.481023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.481300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.482145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=3505cab8f3094ff785667548ec5311ca) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:26.482456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:26.483951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:26.487459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 187/368] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=fed756b9361747358b9a5948eb9a1442 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:26.488285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.491059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 182/369] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=fed756b9361747358b9a5948eb9a1442 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:26.494611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.664796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656adc2-850a-4ffa-8635-8bcb8e902b90 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.682260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:26.700864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ed08-449a-425a-b669-f24b5b38741c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.717616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.718372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.718657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.719218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.720680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:26.720994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:26.724393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:26.725101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:26.742200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 188/370] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:26.743939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.747157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.747929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.748223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.748508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.749539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:26.749644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:26.750966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:26.760263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 183/371] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:26.765764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.862234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52244e-75a0-4a37-80bd-beb352f9cd95 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:26.903359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:26.904407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:26.904915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:26.905390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:26.906570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] RBAC: Adding query filter params (role.id=2f177b995bd14bb4b85362a4f62b975d, user.id=190ffdd855f74fd4a38c4c2131a25e68, scope.system=all) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:26.907068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:26.909232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:26.924454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 189/372] 10.222.0.87 () {56 vars in 1308 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/role_assignments?user.id=190ffdd855f74fd4a38c4c2131a25e68&scope.system=all&role.id=2f177b995bd14bb4b85362a4f62b975d => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:26.926153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:26.967451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:27.022219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae67d-688f-497e-ac4a-13ab2b356149 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:27.123013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:27.124030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:27.124547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:27.125014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:27.129088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d877f-d62e-47d7-87d8-6b820aa4bba6 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:27.223696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:27.224714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:27.225231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:27.225698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:27.231494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:27.265031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:27.266522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:27.384852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 184/373] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:27.390294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:27.394027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:27.396112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:27.489964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:27.498830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:27.518407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9786a-4ad7-4c72-8479-d09062b23677 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:27.556798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:27.557686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:27.557971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:27.558250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:27.560443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:27.561910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:27.566214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 185/374] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:27 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:27.576591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:27.631224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 190/375] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:27.637949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:27.711951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851e1ee-d5bb-4514-b93e-ccb3effabd47 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:27.755329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:27.756273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:27.756561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:27.756843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:27.758170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:27.759793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:27.761543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 186/376] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:27 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:27.783032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:27.823106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:27.876421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b3c07-9fab-47eb-a2f8-b2f328e58e6d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:27.916389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90cdd0-753f-4a8f-a4af-c1bd08a8bd83 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:27.952164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:27.952835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:27.953120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:27.953397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:27.954253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=cd408398a4fd4a1d970960d255e9e000) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:27.954571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:27.959403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:27.962340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 187/377] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:27 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=20865fed4c894cccb7f1c223e95353ec => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:27.974348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a77930-0a4b-4a2c-a74d-033e223b8bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:27.976012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77930-0a4b-4a2c-a74d-033e223b8bc9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:27.976496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77930-0a4b-4a2c-a74d-033e223b8bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:27.976990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77930-0a4b-4a2c-a74d-033e223b8bc9 None admin] PATH_INFO: `/v3/projects/a8cba89e292b47ef9da2168f3c55b232` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:27.980256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a77930-0a4b-4a2c-a74d-033e223b8bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=a8cba89e292b47ef9da2168f3c55b232)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:27.982095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a77930-0a4b-4a2c-a74d-033e223b8bc9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:27.986722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 191/378] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:27 2022] GET /identity/v3/projects/a8cba89e292b47ef9da2168f3c55b232 => generated 344 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:27.990146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:28.108423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bd23-e157-4d8c-a729-c8e363849813 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.108620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bd23-e157-4d8c-a729-c8e363849813 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.108892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bd23-e157-4d8c-a729-c8e363849813 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.109790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:28.110568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 192/379] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:28.117394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfd902-5dcb-4660-9ff6-cb9c6e8d1826 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.117765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfd902-5dcb-4660-9ff6-cb9c6e8d1826 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.118756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfd902-5dcb-4660-9ff6-cb9c6e8d1826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.127813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:28.140209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-35bfd902-5dcb-4660-9ff6-cb9c6e8d1826 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:28.141440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-35bfd902-5dcb-4660-9ff6-cb9c6e8d1826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:28.142271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aaac8-ebd7-43e1-9da9-3cf52538a882 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:28.206754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:28.207827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.208109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.208437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] PATH_INFO: `/v3/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.215788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] RBAC: Authorizing `identity:get_group(group_id=4a2f2f9e68a940569f0a2f0f60b9cdfe)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:28.217681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:28.221288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 188/380] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:27 2022] GET /identity/v3/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:28.227956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:28.249294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 193/381] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:28.272732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfd902-5dcb-4660-9ff6-cb9c6e8d1826 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:28.361037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f6cad-6221-48ce-9f7c-3d5a241337b5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:28.398468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfd902-5dcb-4660-9ff6-cb9c6e8d1826 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:28.399827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:28.400502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.400785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.401064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.401913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] RBAC: Adding query filter params (group.id=4a2f2f9e68a940569f0a2f0f60b9cdfe, role.id=4a6b9a3327544100929540645a797d97, scope.project.id=a8cba89e292b47ef9da2168f3c55b232) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:28.402227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:28.403729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:28.409678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 189/382] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:28 2022] GET /identity/v3/role_assignments?group.id=4a2f2f9e68a940569f0a2f0f60b9cdfe&scope.project.id=a8cba89e292b47ef9da2168f3c55b232&role.id=4a6b9a3327544100929540645a797d97 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:28.459408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:28.463885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:28.465081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.465562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.466053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.489889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:28.498886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:28.504222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:28.504999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:28.507938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 194/383] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:28.516741 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:28.532785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:28.660452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1547595-5387-46f7-b249-9619b3f4d73b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:28.699147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:28.699880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.700164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.700447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.701215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:28.701524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:28.702930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:28.712527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 190/384] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:28.718145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:28.842960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d4e91-9799-4693-97b7-92ec79e357f9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:28.882859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693c03c-d65c-4c05-8755-602c8a3e96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:28.883831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693c03c-d65c-4c05-8755-602c8a3e96b2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.884118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693c03c-d65c-4c05-8755-602c8a3e96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.884392 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693c03c-d65c-4c05-8755-602c8a3e96b2 None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.885803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693c03c-d65c-4c05-8755-602c8a3e96b2 None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:28.887263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693c03c-d65c-4c05-8755-602c8a3e96b2 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:28.889274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 195/385] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:28 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:28.894681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:28.981068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0fb8f-62bd-48ab-8e07-a6c58451f6a9 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.981368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0fb8f-62bd-48ab-8e07-a6c58451f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.981544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0fb8f-62bd-48ab-8e07-a6c58451f6a9 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:28.983583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 196/386] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:28.988262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de6420-9eed-4216-8bef-fc67168ce1d2 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:28.988545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de6420-9eed-4216-8bef-fc67168ce1d2 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:28.988831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de6420-9eed-4216-8bef-fc67168ce1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.003778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:29.021208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-12de6420-9eed-4216-8bef-fc67168ce1d2 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:29.022412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-12de6420-9eed-4216-8bef-fc67168ce1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:29.036221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336f8de-47f3-42aa-8738-3b1707ab3139 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:29.106077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:29.122907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:29.123838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.124122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.124392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.128976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:29.130449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:29.132232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 191/387] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:28 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:29.138672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 197/388] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:29.139960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:29.161777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de6420-9eed-4216-8bef-fc67168ce1d2 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:29.290555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2df61-58d0-405a-afb6-ffbdf466f66d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:29.337266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b876c-b09b-4160-bc35-57e3fea44afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:29.338185 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b876c-b09b-4160-bc35-57e3fea44afc None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.338466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b876c-b09b-4160-bc35-57e3fea44afc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.338739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b876c-b09b-4160-bc35-57e3fea44afc None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca/groups/cbdb84dec13648418926f9a048c799e2/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.343224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b876c-b09b-4160-bc35-57e3fea44afc None admin] RBAC: Authorizing `identity:create_grant(project_id=3505cab8f3094ff785667548ec5311ca, group_id=cbdb84dec13648418926f9a048c799e2, role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:29.344713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b876c-b09b-4160-bc35-57e3fea44afc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:29.365460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 192/389] 10.222.0.87 () {58 vars in 1442 bytes} [Fri Jun 17 12:38:29 2022] PUT /identity/v3/projects/3505cab8f3094ff785667548ec5311ca/groups/cbdb84dec13648418926f9a048c799e2/roles/20865fed4c894cccb7f1c223e95353ec => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:29.367686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:29.378324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85142f-4c41-4cd6-be14-c598b3a95a1d None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.378641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85142f-4c41-4cd6-be14-c598b3a95a1d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.378865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85142f-4c41-4cd6-be14-c598b3a95a1d None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.380600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 193/390] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:29.381145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de6420-9eed-4216-8bef-fc67168ce1d2 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:29.389332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5f6f9-c610-45f3-9b37-888a3e16a3be None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.389645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5f6f9-c610-45f3-9b37-888a3e16a3be None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.389894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5f6f9-c610-45f3-9b37-888a3e16a3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.416726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f4a5f6f9-c610-45f3-9b37-888a3e16a3be None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:29.417933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f4a5f6f9-c610-45f3-9b37-888a3e16a3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:29.468347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:29.469196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.469487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.469762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.498654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:29.506752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:29.507879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:29.508808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 198/391] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:29.517653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:29.523019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 194/392] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:29.550649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5f6f9-c610-45f3-9b37-888a3e16a3be None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:29.587487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:29.644099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd847f-2de9-4d1a-8b25-683e0dbffba5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:29.688178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:29.688869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.689152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.689423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.690183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:29.690495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:29.691930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:29.709600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 199/393] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:29.715086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:29.716316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5f6f9-c610-45f3-9b37-888a3e16a3be None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:29.772144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:29.787631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:29.790010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.791649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.791941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.815300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:29.830235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:29.831160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:29.831982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 195/394] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:29.839245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:29.915011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e6dbe-6885-404c-b983-c84e54b14cd3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:29.944253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:29.967563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:29.968597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:29.968884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:29.969160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:29.970353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:29.971847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:29.974151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 200/395] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:29 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:29.980944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:30.040036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eca684-11f8-4958-8cc1-b13b7bd1f6ec None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:30.108488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:30.109084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.119305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.119666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.120487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] RBAC: Adding query filter params (name=member) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:30.120805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:30.122231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:30.142765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 196/396] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:30.147210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:30.197423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874359d-1112-482e-ae46-8f4e8bddb01a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:30.207489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:30.262064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf03b96-acd2-4aef-98a5-b796d5820c66 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:30.300910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:30.303052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.303925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.304670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.307125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:30.307451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:30.308569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.310183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:30.310612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.311027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.313322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:30.314916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:30.316139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 201/397] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:29 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:30.318317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 197/398] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:30 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:30.326607 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:30.326767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:30.383201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:30.470533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb73092-a605-486f-aaa3-0c8d876882f4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:30.499095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff73030-bdcc-42c7-b6ba-0e762f62cadf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:30.518089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:30.530797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc60e5c-4949-47c3-bd0c-36d60ff2ac8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:30.531769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60e5c-4949-47c3-bd0c-36d60ff2ac8d None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.532054 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60e5c-4949-47c3-bd0c-36d60ff2ac8d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.532332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60e5c-4949-47c3-bd0c-36d60ff2ac8d None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000/groups/cbdb84dec13648418926f9a048c799e2/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.534417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc60e5c-4949-47c3-bd0c-36d60ff2ac8d None admin] RBAC: Authorizing `identity:create_grant(project_id=cd408398a4fd4a1d970960d255e9e000, group_id=cbdb84dec13648418926f9a048c799e2, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:30.536017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc60e5c-4949-47c3-bd0c-36d60ff2ac8d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:30.537251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:30.538128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.538415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.538746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.544228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:30.545695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:30.547919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 198/399] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:30 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:30.559265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:30.561148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 202/400] 10.222.0.87 () {58 vars in 1442 bytes} [Fri Jun 17 12:38:30 2022] PUT /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000/groups/cbdb84dec13648418926f9a048c799e2/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:30.679957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dcf2c-9083-4242-be4f-4dd7dacc1cad None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:30.722756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fc431-6849-427a-828f-5e0237f2e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:30.723724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fc431-6849-427a-828f-5e0237f2e3bc None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.724010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fc431-6849-427a-828f-5e0237f2e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.724289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fc431-6849-427a-828f-5e0237f2e3bc None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca/groups/cbdb84dec13648418926f9a048c799e2/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.726369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fc431-6849-427a-828f-5e0237f2e3bc None admin] RBAC: Authorizing `identity:create_grant(project_id=3505cab8f3094ff785667548ec5311ca, group_id=cbdb84dec13648418926f9a048c799e2, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:30.727926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fc431-6849-427a-828f-5e0237f2e3bc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:30.746000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 199/401] 10.222.0.87 () {58 vars in 1442 bytes} [Fri Jun 17 12:38:30 2022] PUT /identity/v3/projects/3505cab8f3094ff785667548ec5311ca/groups/cbdb84dec13648418926f9a048c799e2/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:30.897734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d2c7-a120-4aca-b42c-cc5f55073818 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.898135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d2c7-a120-4aca-b42c-cc5f55073818 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.898324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d2c7-a120-4aca-b42c-cc5f55073818 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.900252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 203/402] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:30.906539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc859ab-f5eb-419b-895e-3604745a9a72 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:30.906825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc859ab-f5eb-419b-895e-3604745a9a72 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:30.907166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc859ab-f5eb-419b-895e-3604745a9a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:30.925651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-5fc859ab-f5eb-419b-895e-3604745a9a72 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:30.926956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5fc859ab-f5eb-419b-895e-3604745a9a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:30.980018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:31.007456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 200/403] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:31.025018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190febb-9f1c-4aca-af21-d73149669480 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.025298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190febb-9f1c-4aca-af21-d73149669480 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.025583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190febb-9f1c-4aca-af21-d73149669480 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.027242 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 204/404] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:31.029811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc859ab-f5eb-419b-895e-3604745a9a72 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.043938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf5ea-bd5b-405f-a818-d0aefdc86014 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.044229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf5ea-bd5b-405f-a818-d0aefdc86014 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.044508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf5ea-bd5b-405f-a818-d0aefdc86014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.075028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e87bf5ea-bd5b-405f-a818-d0aefdc86014 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:31.077669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e87bf5ea-bd5b-405f-a818-d0aefdc86014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:31.183244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 205/405] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:31.207436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf5ea-bd5b-405f-a818-d0aefdc86014 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.216065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc859ab-f5eb-419b-895e-3604745a9a72 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:31.253492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:31.254722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.255270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.255774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.280316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:31.287702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:31.294224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:31.295109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:31.295929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:31.299584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 201/406] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:31.307765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.370898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:31.393353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf5ea-bd5b-405f-a818-d0aefdc86014 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:31.452349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f6e41-43db-4dfb-b1b3-04357a5ce364 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:31.488936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:31.489745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.490033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.490309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.491173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:31.491516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:31.493151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:31.502230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 202/407] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:31.507682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.512077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:31.513450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.514000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.514449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.567304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:31.583582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:31.584481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:31.585265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:31.587705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 206/408] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:31.593676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.607793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:31.619670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aabf1-311e-4098-82bf-8ee956eb2cbc None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:31.679352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:31.680556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.680742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.680991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.682219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:31.691846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:31.693946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 203/409] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:31 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:31.699580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.707593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da6d3-a355-4001-84fe-f6cf9403e525 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:31.719750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:31.751530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:31.752622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.753128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.753628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.754678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:31.755186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:31.756803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:31.775294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 207/410] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:31.776656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.827545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1118-e1bc-4124-97da-5a3f50b28e0a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:31.862328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccdf4d-b639-404e-95ea-fbeffc782141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:31.863206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccdf4d-b639-404e-95ea-fbeffc782141 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.863521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccdf4d-b639-404e-95ea-fbeffc782141 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.863799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccdf4d-b639-404e-95ea-fbeffc782141 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.864991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccdf4d-b639-404e-95ea-fbeffc782141 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:31.866457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccdf4d-b639-404e-95ea-fbeffc782141 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:31.868670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 204/411] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:31 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:31.869854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd7cb-330e-4393-94a1-10b218cd0821 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.870148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd7cb-330e-4393-94a1-10b218cd0821 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.870417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd7cb-330e-4393-94a1-10b218cd0821 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.876456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 205/412] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:31.877193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd7cb-330e-4393-94a1-10b218cd0821 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:31.919691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:31.933284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563240-f2ea-4f12-80b2-e12b08121db9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:31.977861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e6af6-675f-4822-b035-16f5bb6e690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:31.979096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6af6-675f-4822-b035-16f5bb6e690f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.979795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6af6-675f-4822-b035-16f5bb6e690f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.980617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6af6-675f-4822-b035-16f5bb6e690f None admin] PATH_INFO: `/v3/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.982184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e6af6-675f-4822-b035-16f5bb6e690f None admin] RBAC: Authorizing `identity:get_group(group_id=4a2f2f9e68a940569f0a2f0f60b9cdfe)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:31.984271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e6af6-675f-4822-b035-16f5bb6e690f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:31.990662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 208/413] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:31 2022] GET /identity/v3/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:31.990662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520781c8-e3f0-4e51-b958-e88278363201 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:31.990662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520781c8-e3f0-4e51-b958-e88278363201 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:31.990662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520781c8-e3f0-4e51-b958-e88278363201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:31.997406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd7cb-330e-4393-94a1-10b218cd0821 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:32.013103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-520781c8-e3f0-4e51-b958-e88278363201 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:32.014854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-520781c8-e3f0-4e51-b958-e88278363201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:32.042135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:32.043125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:32.043540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:32.043820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000/groups/cbdb84dec13648418926f9a048c799e2/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:32.045435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:32.046214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] RBAC: Authorizing `identity:create_grant(project_id=cd408398a4fd4a1d970960d255e9e000, group_id=cbdb84dec13648418926f9a048c799e2, role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:32.048055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:32.067452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 206/414] 10.222.0.87 () {58 vars in 1442 bytes} [Fri Jun 17 12:38:31 2022] PUT /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000/groups/cbdb84dec13648418926f9a048c799e2/roles/20865fed4c894cccb7f1c223e95353ec => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:32.068677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:32.141093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 209/415] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:32.202766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520781c8-e3f0-4e51-b958-e88278363201 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:32.239176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:32.249775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee66519-3d36-437d-9680-ed136cbb8c7b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:32.314846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:32.315719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:32.316004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:32.316280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] PATH_INFO: `/v3/projects/a8cba89e292b47ef9da2168f3c55b232` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:32.317478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] RBAC: Authorizing `identity:get_project(project_id=a8cba89e292b47ef9da2168f3c55b232)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:32.318928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:32.325445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 207/416] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:32 2022] GET /identity/v3/projects/a8cba89e292b47ef9da2168f3c55b232 => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:32.327057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:32.367244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520781c8-e3f0-4e51-b958-e88278363201 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:32.413999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:32.415772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:32.416254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:32.416722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:32.431188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:32.443230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:32.444115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:32.444906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:32.445500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214777e-eb96-4164-9a7b-ad2104711fbf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:32.448236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 210/417] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:32.457022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:32.490949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:32.512209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:32.513128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:32.513414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:32.513694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] PATH_INFO: `/v3/projects/a8cba89e292b47ef9da2168f3c55b232/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:32.517293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8cba89e292b47ef9da2168f3c55b232, group_id=4a2f2f9e68a940569f0a2f0f60b9cdfe, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:32.519672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:32.547532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 208/418] 10.222.0.87 () {58 vars in 1442 bytes} [Fri Jun 17 12:38:32 2022] PUT /identity/v3/projects/a8cba89e292b47ef9da2168f3c55b232/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:32.604855 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9826f-d725-4bfb-afbf-68ecceb639e5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:32.656183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:32.656871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:32.657160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:32.657442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:32.658210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:32.658523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:32.660026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:32.671817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 211/419] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:32.677362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:32.710915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:32.819208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2e431-d964-4f4b-b5fc-f0fb78a45ff6 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:32.856710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:32.857304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:32.857624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:32.857914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:32.941424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:32.942913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:32.969578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:33.071277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 209/420] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:33.076863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:33.227707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:33.253216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512acc0-3166-4ccf-aad3-046f930014d9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:33.308364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ed0a5-2b6e-43f5-b9d7-83952ba2b000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:33.309299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ed0a5-2b6e-43f5-b9d7-83952ba2b000 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.309584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ed0a5-2b6e-43f5-b9d7-83952ba2b000 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.309865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ed0a5-2b6e-43f5-b9d7-83952ba2b000 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.311305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ed0a5-2b6e-43f5-b9d7-83952ba2b000 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:33.312854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ed0a5-2b6e-43f5-b9d7-83952ba2b000 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:33.315045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 212/421] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:33 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:33.320550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:33.375326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57752729-97c9-441f-ac96-7776b9c2a78a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.375622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57752729-97c9-441f-ac96-7776b9c2a78a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.375845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57752729-97c9-441f-ac96-7776b9c2a78a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.377464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 213/422] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:33.382485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aa20a-7a06-41c1-9e18-384b4cd7f0de None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.382787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aa20a-7a06-41c1-9e18-384b4cd7f0de None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.383058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aa20a-7a06-41c1-9e18-384b4cd7f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.400380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e63aa20a-7a06-41c1-9e18-384b4cd7f0de None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:33.401567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e63aa20a-7a06-41c1-9e18-384b4cd7f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:33.467856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fca092-b689-47d4-bfd4-9e25cea99df7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:33.477195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 214/423] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:33.483522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de005ee2-e16c-4a6c-a738-a6a8d216317a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.484312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de005ee2-e16c-4a6c-a738-a6a8d216317a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.484683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de005ee2-e16c-4a6c-a738-a6a8d216317a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.487385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 215/424] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:33.492697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f4134-cff4-4c47-9587-0215ffc366fc None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.492959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f4134-cff4-4c47-9587-0215ffc366fc None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.493285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f4134-cff4-4c47-9587-0215ffc366fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.494698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:33.519124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:33.520361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.520657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-604f4134-cff4-4c47-9587-0215ffc366fc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:33.520976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.521391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.522190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-604f4134-cff4-4c47-9587-0215ffc366fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:33.522856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:33.524762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:33.526809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 210/425] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:33 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:33.527744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:33.607641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 216/426] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:33.609224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f4134-cff4-4c47-9587-0215ffc366fc None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:33.619894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:33.656755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2145c67-69d7-46f0-a070-ef4b98cfa740 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:33.702097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:33.703097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.703477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.703817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.718067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f4134-cff4-4c47-9587-0215ffc366fc None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:33.718259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:33.727691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:33.728698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:33.729494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:33.731447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 211/427] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:33.732475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:33.768427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:33.769358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.769651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.769932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.770843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=3505cab8f3094ff785667548ec5311ca) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:33.771165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:33.772927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:33.780818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 217/428] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:33 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=20865fed4c894cccb7f1c223e95353ec => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:33.782155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:33.803887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:33.887837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:33.909794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f9b3-2aa3-4e5f-8a07-c56f5deac31f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:33.950027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:33.950720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:33.951005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:33.951281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:33.952644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] RBAC: Adding query filter params (name=member) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:33.953092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:33.954490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:33.954828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6701d-6c84-4ca5-b5ac-86d6d3330be7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:33.967471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 218/429] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:33.972622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:33.999874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.000751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.001036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.001310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.013604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:34.022594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:34.023531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:34.024390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 212/430] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:34.030038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.109476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec3447-d71e-42c5-8648-75b20ae45b2a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:34.156949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.157620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.157903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.158182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.161475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:34.197486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79feb-7be1-4dbe-b0fa-30126969e0e2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:34.211926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:34.244313 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:34.244748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.245648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.245820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:34.246282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.246771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.248101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:34.249529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:34.251121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:34.263153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 213/431] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:34.268167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.366470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 219/432] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:34.367874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.401441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:34.406775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed35e4-65f1-4ca3-971f-25026c5f78e5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:34.445910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf6304-d182-431d-8eb6-08bad7d763a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.446513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf6304-d182-431d-8eb6-08bad7d763a8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.446805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf6304-d182-431d-8eb6-08bad7d763a8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.447078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf6304-d182-431d-8eb6-08bad7d763a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.487599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d9b8a-4ed1-4f31-adb9-7a28bf1deace None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:34.489126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:34.524829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf6304-d182-431d-8eb6-08bad7d763a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:34.526467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.527353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.527642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf6304-d182-431d-8eb6-08bad7d763a8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:34.527742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.527924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] PATH_INFO: `/v3/projects/3505cab8f3094ff785667548ec5311ca` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.529139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] RBAC: Authorizing `identity:get_project(project_id=3505cab8f3094ff785667548ec5311ca)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:34.530700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:34.538278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 220/433] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/projects/3505cab8f3094ff785667548ec5311ca => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:34.540320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.652524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b964cb-cf28-4d31-9b2d-362f8968a6e0 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:34.653875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:34.670018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 214/434] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 12:38:34.671638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c1c7-6965-48ee-8f4c-c61e52c3db63 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.671929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c1c7-6965-48ee-8f4c-c61e52c3db63 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.672205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c1c7-6965-48ee-8f4c-c61e52c3db63 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.673811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 215/435] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:34.674946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c1c7-6965-48ee-8f4c-c61e52c3db63 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.691162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421304e-5457-4082-9da8-21b5fda7d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.692420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421304e-5457-4082-9da8-21b5fda7d13f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.692858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421304e-5457-4082-9da8-21b5fda7d13f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.693131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421304e-5457-4082-9da8-21b5fda7d13f None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.694331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421304e-5457-4082-9da8-21b5fda7d13f None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:34.695832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421304e-5457-4082-9da8-21b5fda7d13f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:34.702321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 221/436] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:34.702321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd5e10-7fb6-42e8-8a37-4fb96135414d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.702321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd5e10-7fb6-42e8-8a37-4fb96135414d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.702321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd5e10-7fb6-42e8-8a37-4fb96135414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.719794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-18dd5e10-7fb6-42e8-8a37-4fb96135414d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:34.720975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-18dd5e10-7fb6-42e8-8a37-4fb96135414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:34.756615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:34.790173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 222/437] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:34.791116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd5e10-7fb6-42e8-8a37-4fb96135414d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.800607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c1c7-6965-48ee-8f4c-c61e52c3db63 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:34.842239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.843688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.844198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.844671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.846147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:34.848197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:34.850213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 216/438] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:34.851165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.901551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:34.905388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd5e10-7fb6-42e8-8a37-4fb96135414d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:34.943723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:34.944526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:34.944792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:34.945069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:34.945909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=3505cab8f3094ff785667548ec5311ca) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:34.946286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:34.947898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:34.963684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 223/439] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=3505cab8f3094ff785667548ec5311ca&role.id=fed756b9361747358b9a5948eb9a1442 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:34.963684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:34.968681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7249ff-bf08-45ae-910f-c90deb6f7bd8 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.009232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.010747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.011034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.011312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.022264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.026298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:35.038391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:35.039341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:35.040622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:35.043619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 217/440] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:35.046359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.085221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab718cf5-8779-4e08-a4d0-f173cb670fcf None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.138153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d7d16-bee8-4bbf-8c36-5022c1ce827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.139083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d7d16-bee8-4bbf-8c36-5022c1ce827d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.139400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d7d16-bee8-4bbf-8c36-5022c1ce827d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.139871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d7d16-bee8-4bbf-8c36-5022c1ce827d None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.141572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d7d16-bee8-4bbf-8c36-5022c1ce827d None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:35.143858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d7d16-bee8-4bbf-8c36-5022c1ce827d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:35.147784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 224/441] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:34 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:35.151953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873233d-e8cc-4d7e-ae92-703eedfe363c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.152522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873233d-e8cc-4d7e-ae92-703eedfe363c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.152871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873233d-e8cc-4d7e-ae92-703eedfe363c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.154509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 225/442] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:35.155162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873233d-e8cc-4d7e-ae92-703eedfe363c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.192248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.203527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b324b-a901-41eb-a800-c5baf5485b19 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.250632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dfffe-d27c-4323-928d-949db62af370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.251736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dfffe-d27c-4323-928d-949db62af370 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.252028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dfffe-d27c-4323-928d-949db62af370 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.252308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dfffe-d27c-4323-928d-949db62af370 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.253239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dfffe-d27c-4323-928d-949db62af370 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:35.253936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dfffe-d27c-4323-928d-949db62af370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:35.255635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dfffe-d27c-4323-928d-949db62af370 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:35.264446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 218/443] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:35.265791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49335b29-d0ae-4517-a16c-816a8dafc314 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.266092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49335b29-d0ae-4517-a16c-816a8dafc314 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.266371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49335b29-d0ae-4517-a16c-816a8dafc314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.276297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873233d-e8cc-4d7e-ae92-703eedfe363c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.285249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-49335b29-d0ae-4517-a16c-816a8dafc314 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:35.286854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-49335b29-d0ae-4517-a16c-816a8dafc314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:35.318811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.319364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.319730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.320004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.320851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=fed756b9361747358b9a5948eb9a1442, scope.project.id=cd408398a4fd4a1d970960d255e9e000) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:35.321498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:35.322957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:35.329425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 226/444] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=fed756b9361747358b9a5948eb9a1442 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:35.330070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.364381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.382088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 219/445] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:35.406811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49335b29-d0ae-4517-a16c-816a8dafc314 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.476583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.486866 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bfb1b-fc64-4c6d-93f7-3367f7a5d157 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.530552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.531897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.532192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.532475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.546576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49335b29-d0ae-4517-a16c-816a8dafc314 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.599588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.600414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.601290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.601290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.612582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:35.614790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:35.616579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:35.620728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:35.621822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:35.622852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 220/446] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:35.629843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.687556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.723546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.732713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 227/447] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:35.738455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.739903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5360e9-6d04-4e82-b961-f8c58179cb70 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.777659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.778470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.778818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.779161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.780052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:35.780424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:35.781946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:35.791283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 221/448] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:35.796799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.844200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b17f30-315a-4bc3-8069-56a013316bdb None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.877029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.877901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.878186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.878465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] PATH_INFO: `/v3/projects/cd408398a4fd4a1d970960d255e9e000` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.879696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] RBAC: Authorizing `identity:get_project(project_id=cd408398a4fd4a1d970960d255e9e000)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:35.881200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:35.882906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 228/449] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/projects/cd408398a4fd4a1d970960d255e9e000 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:35.888911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:35.894636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b0c3f-a26d-46b7-94fa-dc70193935c0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.908952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.930332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:35.931064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:35.931412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:35.931697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:35.950507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:35.986639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79980978-2cb4-4a35-8326-ebe98fab3fed None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:35.996437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:35.997910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:36.019974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:36.020842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:36.021130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:36.021406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] PATH_INFO: `/v3/groups/cbdb84dec13648418926f9a048c799e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:36.022597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] RBAC: Authorizing `identity:get_group(group_id=cbdb84dec13648418926f9a048c799e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:36.024088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:36.026092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 229/450] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/groups/cbdb84dec13648418926f9a048c799e2 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:36.032192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:36.095539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 222/451] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 12:38:36.101093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:36.117465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:36.131600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819bdd2-bb6e-4a7f-a575-b16f2c56e2fc None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:36.160652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:36.166350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:36.167011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:36.167293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:36.167605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:36.168448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] RBAC: Adding query filter params (group.id=cbdb84dec13648418926f9a048c799e2, role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=cd408398a4fd4a1d970960d255e9e000) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:36.168762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:36.170214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:36.176271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 230/452] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:36 2022] GET /identity/v3/role_assignments?group.id=cbdb84dec13648418926f9a048c799e2&scope.project.id=cd408398a4fd4a1d970960d255e9e000&role.id=20865fed4c894cccb7f1c223e95353ec => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:36.205822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3e9ca-7709-48d5-b65f-923d951d57e5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:36.241984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:36.242867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:36.243273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:36.243607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] PATH_INFO: `/v3/projects/a8cba89e292b47ef9da2168f3c55b232` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:36.244878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] RBAC: Authorizing `identity:get_project(project_id=a8cba89e292b47ef9da2168f3c55b232)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:36.246348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:36.248160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 223/453] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:38:36 2022] GET /identity/v3/projects/a8cba89e292b47ef9da2168f3c55b232 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:36.253997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:36.364252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08a093-0730-43d7-a177-3f46d5f59f99 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:36.397516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca83e7-b795-420e-a0b2-bc10293883e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:36.398661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca83e7-b795-420e-a0b2-bc10293883e1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:36.398942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca83e7-b795-420e-a0b2-bc10293883e1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:36.399214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca83e7-b795-420e-a0b2-bc10293883e1 None admin] PATH_INFO: `/v3/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:36.400592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca83e7-b795-420e-a0b2-bc10293883e1 None admin] RBAC: Authorizing `identity:get_group(group_id=4a2f2f9e68a940569f0a2f0f60b9cdfe)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:36.402052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca83e7-b795-420e-a0b2-bc10293883e1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:36.403726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 231/454] 10.222.0.87 () {56 vars in 1180 bytes} [Fri Jun 17 12:38:36 2022] GET /identity/v3/groups/4a2f2f9e68a940569f0a2f0f60b9cdfe => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:36.409909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:36.492391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:36.511551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2687e30-35a8-4433-b843-a9d0deb583ca None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:36.546359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4c718-002e-4497-b6f5-e06059912b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:36.547021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4c718-002e-4497-b6f5-e06059912b86 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:36.547308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4c718-002e-4497-b6f5-e06059912b86 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:36.547631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4c718-002e-4497-b6f5-e06059912b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:36.548480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4c718-002e-4497-b6f5-e06059912b86 None admin] RBAC: Adding query filter params (group.id=4a2f2f9e68a940569f0a2f0f60b9cdfe, role.id=4a6b9a3327544100929540645a797d97, scope.project.id=a8cba89e292b47ef9da2168f3c55b232) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:36.548795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4c718-002e-4497-b6f5-e06059912b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:36.550245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4c718-002e-4497-b6f5-e06059912b86 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:36.556106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 224/455] 10.222.0.87 () {56 vars in 1376 bytes} [Fri Jun 17 12:38:36 2022] GET /identity/v3/role_assignments?group.id=4a2f2f9e68a940569f0a2f0f60b9cdfe&scope.project.id=a8cba89e292b47ef9da2168f3c55b232&role.id=4a6b9a3327544100929540645a797d97 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:39.129333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e839f3-b4ac-47a6-b4dc-7bac9dbfe05a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.129847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e839f3-b4ac-47a6-b4dc-7bac9dbfe05a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.129955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e839f3-b4ac-47a6-b4dc-7bac9dbfe05a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.131271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 232/456] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:39.140633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee507d-adb4-4df9-bf7a-c375548bd6b5 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.140633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee507d-adb4-4df9-bf7a-c375548bd6b5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.140633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee507d-adb4-4df9-bf7a-c375548bd6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.163541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-19ee507d-adb4-4df9-bf7a-c375548bd6b5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:39.165474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-19ee507d-adb4-4df9-bf7a-c375548bd6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:39.271140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 225/457] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:39.288746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eee78f-bc43-4581-b86b-12ef439d2553 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.289312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eee78f-bc43-4581-b86b-12ef439d2553 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.290168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eee78f-bc43-4581-b86b-12ef439d2553 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.292078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 233/458] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:39.297790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147607d-c3b4-4dc3-bb55-bae2c5d253c4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.298301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147607d-c3b4-4dc3-bb55-bae2c5d253c4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.298773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147607d-c3b4-4dc3-bb55-bae2c5d253c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.314300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eee78f-bc43-4581-b86b-12ef439d2553 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:39.328187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3147607d-c3b4-4dc3-bb55-bae2c5d253c4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:39.330781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3147607d-c3b4-4dc3-bb55-bae2c5d253c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:39.424611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:39.461048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 226/459] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:39.488052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eee78f-bc43-4581-b86b-12ef439d2553 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:39.516865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147607d-c3b4-4dc3-bb55-bae2c5d253c4 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:39.531122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:39.548360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:39.549528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.549989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.550439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.574096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:39.576272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:39.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:39.608209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:39.609053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:39.609053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:39.609053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:39.609053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:39.609053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 234/460] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:39.616469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:39.653201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147607d-c3b4-4dc3-bb55-bae2c5d253c4 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:39.693405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:39.694636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.695148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.695704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.708735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:39.710769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:39.731791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:39.732703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:39.733516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:39.733516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:39.733516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:39.733516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:39.735693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 227/461] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:39.742002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:39.804243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50bf1b-2078-41ac-950a-43e1f54431f8 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:39.813322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:39.826222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:39.850741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b7569-2499-4614-ba5c-d5b2c418ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:39.852387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7569-2499-4614-ba5c-d5b2c418ad19 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.852892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7569-2499-4614-ba5c-d5b2c418ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.853579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7569-2499-4614-ba5c-d5b2c418ad19 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.854583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b7569-2499-4614-ba5c-d5b2c418ad19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:39.855043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b7569-2499-4614-ba5c-d5b2c418ad19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:39.856938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b7569-2499-4614-ba5c-d5b2c418ad19 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:39.861504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e0179-77a5-4242-8e54-122a13ac7689 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:39.886072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 235/462] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:39.886072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7a84-8f98-4646-8e91-963cb08a9c3a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.886072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7a84-8f98-4646-8e91-963cb08a9c3a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.886072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7a84-8f98-4646-8e91-963cb08a9c3a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.886072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 236/463] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:39.886072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789da5-7db0-4250-8cfd-c9ddb3a45722 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.886579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789da5-7db0-4250-8cfd-c9ddb3a45722 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.886579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789da5-7db0-4250-8cfd-c9ddb3a45722 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.890151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 237/464] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:39.890352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa5714-b352-40fc-b344-45262238596a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.890636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa5714-b352-40fc-b344-45262238596a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.890914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa5714-b352-40fc-b344-45262238596a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.909193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9ffa5714-b352-40fc-b344-45262238596a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:39.911358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:39.921168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9ffa5714-b352-40fc-b344-45262238596a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:39.921342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:39.921342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:39.921342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:39.921342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:39.921342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:39.921342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:39.942613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 228/465] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:39.942613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.000783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 238/466] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:40.008882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fed1c-7f78-4100-a672-4007664b87d3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.009342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fed1c-7f78-4100-a672-4007664b87d3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.009654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fed1c-7f78-4100-a672-4007664b87d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.031902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-452fed1c-7f78-4100-a672-4007664b87d3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:40.033115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-452fed1c-7f78-4100-a672-4007664b87d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:40.061988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd686e21-e3be-4f5f-ba5a-9ff6b08c82f6 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:40.081302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:40.101554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:40.102236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.102521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.102922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.103779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:40.105244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:40.125389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 239/467] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:40.125876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fed1c-7f78-4100-a672-4007664b87d3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.145102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 229/468] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:39 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:40.147429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.177728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:40.270151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fed1c-7f78-4100-a672-4007664b87d3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:40.310044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578a0ed-f6fc-465b-9f8c-e9a23c113339 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:40.317623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:40.318448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.318799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.319147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.320007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:40.321914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:40.351595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 240/469] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:40 2022] POST /identity/v3/users => generated 248 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:40.352655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.364276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:40.364932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.364932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.365231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.378412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:40.380037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:40.400605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:40.401876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:40.402867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:40.402867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:40.402867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:40.402867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:40.402867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 230/470] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:40.409102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.442683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:40.498126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:40.514336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b9cca-fc98-468a-9498-410b449e6d4d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:40.528884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339e08d-4f3a-43fb-a94a-41e98921687b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:40.574152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:40.575668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.575975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:40.576290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.576743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.576932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.577315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.577845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.580541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:40.581152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:40.582457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:40.592625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:40.595000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:40.598012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 231/471] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:40.605795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:40.618718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:40.619817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:40.620627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:40.620627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:40.620627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:40.620627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:40.627542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 241/472] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:40.630489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.707496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:40.725789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5ec71-191a-495a-9d18-213e55cb6597 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:40.757877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b7be4-ec70-4ab9-9a61-0db416f095e4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:40.766144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:40.766973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.767262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.767606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.768416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:40.769926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:40.801907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 232/473] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:40 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:38:40.805431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd14ad4-acb6-4274-af29-123c3a3f51ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:40.806553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd14ad4-acb6-4274-af29-123c3a3f51ab None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.807111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd14ad4-acb6-4274-af29-123c3a3f51ab None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.807656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd14ad4-acb6-4274-af29-123c3a3f51ab None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.808649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd14ad4-acb6-4274-af29-123c3a3f51ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:40.809171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd14ad4-acb6-4274-af29-123c3a3f51ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:40.810876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd14ad4-acb6-4274-af29-123c3a3f51ab None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:40.825549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 242/474] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:40.831550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:40.867496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95bf28-8c66-4662-86d3-307a72606d63 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.868071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95bf28-8c66-4662-86d3-307a72606d63 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.868541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95bf28-8c66-4662-86d3-307a72606d63 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.870486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 243/475] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:40.881593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c095b-0889-4825-bd5a-8e65fe481461 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:40.882218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c095b-0889-4825-bd5a-8e65fe481461 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:40.882500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c095b-0889-4825-bd5a-8e65fe481461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:40.889448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:40.902367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-274c095b-0889-4825-bd5a-8e65fe481461 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:40.903270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-274c095b-0889-4825-bd5a-8e65fe481461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:40.987042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 244/476] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:40.997362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e3791-c080-4d83-b5be-c3ac05130a5e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:41.049320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:41.050077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:41.050205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c095b-0889-4825-bd5a-8e65fe481461 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:41.050370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:41.050654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:41.051407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:41.052942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:41.071929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:41.078464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 233/477] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:40 2022] POST /identity/v3/users => generated 249 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:41.210402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c095b-0889-4825-bd5a-8e65fe481461 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:41.247345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:41.248097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:41.248380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:41.248702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:41.249473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:41.250886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:41.253464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:41.260647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.sql.core [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 12:38:41.260647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 12:38:41.260647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [parameters: {'id': '97fc4f793cec47ab93028aa730ca26f7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 12:38:41.260647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=97010) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application conn.query(q) Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application result.read() Jun 17 12:38:41.278351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return ref.to_dict() Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 12:38:41.279816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application next(self.gen) Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application yield resource Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application next(self.gen) Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise self.value Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application session.commit() Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 12:38:41.280711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self.session.flush() Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application compat.raise_( Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 12:38:41.281543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 12:38:41.282458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application conn.query(q) Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application result.read() Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 12:38:41.283266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [parameters: {'id': '97fc4f793cec47ab93028aa730ca26f7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:41.286063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 12:38:41.288261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:41.291008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 245/478] 10.222.0.87 () {60 vars in 1131 bytes} [Fri Jun 17 12:38:41 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:41.299586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:41.415336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80a079-1097-4ee7-8efe-d8f9a4fdd67e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:41.454028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2a0a0-b9da-430c-8978-1b35cce87b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:41.454874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a0a0-b9da-430c-8978-1b35cce87b5a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:41.455157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a0a0-b9da-430c-8978-1b35cce87b5a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:41.455462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a0a0-b9da-430c-8978-1b35cce87b5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:41.469957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2a0a0-b9da-430c-8978-1b35cce87b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-95b2a0a0-b9da-430c-8978-1b35cce87b5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:41.478322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:41.479217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 12:38:41.480042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:41.480644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 234/479] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:38:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:41.486931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:41.606193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b78db8-8dcb-4945-aeae-710ed30a6771 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:41.646387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7aa6f4-6844-420b-87cf-270a1452cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:41.647082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7aa6f4-6844-420b-87cf-270a1452cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:41.647381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7aa6f4-6844-420b-87cf-270a1452cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:41.647664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7aa6f4-6844-420b-87cf-270a1452cbf7 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:41.648442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7aa6f4-6844-420b-87cf-270a1452cbf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:41.648749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7aa6f4-6844-420b-87cf-270a1452cbf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:41.650222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7aa6f4-6844-420b-87cf-270a1452cbf7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:41.660479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 246/480] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:43.059470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9738c-3baa-4e7f-a878-96dceb5cd1dd None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.060563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9738c-3baa-4e7f-a878-96dceb5cd1dd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.061051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9738c-3baa-4e7f-a878-96dceb5cd1dd None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.063099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 235/481] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:43.069360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77efe7-5ec3-42d3-bd94-fb00a981fc1a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.070330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77efe7-5ec3-42d3-bd94-fb00a981fc1a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.070801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77efe7-5ec3-42d3-bd94-fb00a981fc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.098632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-fe77efe7-5ec3-42d3-bd94-fb00a981fc1a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:43.099858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fe77efe7-5ec3-42d3-bd94-fb00a981fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:43.129157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79860-49ce-4d9c-810c-64ed3b3bfbbe None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.129445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79860-49ce-4d9c-810c-64ed3b3bfbbe None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.129724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79860-49ce-4d9c-810c-64ed3b3bfbbe None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.131251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 236/482] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:43.136721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef514b-9341-4da8-b778-d5f24a75e430 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.137250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef514b-9341-4da8-b778-d5f24a75e430 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.137726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef514b-9341-4da8-b778-d5f24a75e430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.164229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-23ef514b-9341-4da8-b778-d5f24a75e430 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:43.166369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-23ef514b-9341-4da8-b778-d5f24a75e430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:43.201103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:43.222452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 247/483] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:43.243216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77efe7-5ec3-42d3-bd94-fb00a981fc1a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:43.271775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:43.294991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 237/484] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:43.329287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef514b-9341-4da8-b778-d5f24a75e430 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:43.412479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77efe7-5ec3-42d3-bd94-fb00a981fc1a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:43.456396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:43.457660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.458185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.458701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.482290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:43.491698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:43.492651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:43.493494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:43.506423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 248/485] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:43.513023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:43.542159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef514b-9341-4da8-b778-d5f24a75e430 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:43.571335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:43.579245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3af801-1366-4bed-ba34-3de080ae791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:43.580148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af801-1366-4bed-ba34-3de080ae791a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.580371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af801-1366-4bed-ba34-3de080ae791a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.580644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af801-1366-4bed-ba34-3de080ae791a None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.594885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3af801-1366-4bed-ba34-3de080ae791a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-fc3af801-1366-4bed-ba34-3de080ae791a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:43.604811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:43.605850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:43.606871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 238/486] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:43.607951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7f8f3-78cc-4ee2-ba5f-8f5f8e244c35 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.608239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7f8f3-78cc-4ee2-ba5f-8f5f8e244c35 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.608515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7f8f3-78cc-4ee2-ba5f-8f5f8e244c35 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.609787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 239/487] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:43.614722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7f8f3-78cc-4ee2-ba5f-8f5f8e244c35 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:43.630179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20038b18-449e-45df-b63c-e107b551e773 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:43.664132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837b304-e31a-459b-80f9-e8e1b7c9baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:43.664980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b304-e31a-459b-80f9-e8e1b7c9baf5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.665755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b304-e31a-459b-80f9-e8e1b7c9baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.666131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b304-e31a-459b-80f9-e8e1b7c9baf5 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.666991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b304-e31a-459b-80f9-e8e1b7c9baf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:43.667433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b304-e31a-459b-80f9-e8e1b7c9baf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:43.667851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:43.669014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b304-e31a-459b-80f9-e8e1b7c9baf5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:43.677342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 249/488] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:43.678704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea55fa-bdc6-4f7a-bb3c-fd3dfe30beff None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.679091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea55fa-bdc6-4f7a-bb3c-fd3dfe30beff None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.679492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea55fa-bdc6-4f7a-bb3c-fd3dfe30beff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.696456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-bfea55fa-bdc6-4f7a-bb3c-fd3dfe30beff None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:43.698068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bfea55fa-bdc6-4f7a-bb3c-fd3dfe30beff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:43.729787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7f8f3-78cc-4ee2-ba5f-8f5f8e244c35 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:43.772098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:43.772925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.773382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.773713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.774454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 250/489] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:43.774572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:43.775125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:43.775851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea55fa-bdc6-4f7a-bb3c-fd3dfe30beff None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:43.780609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:43.807487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 240/490] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:43.810311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:43.840974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:43.905191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea55fa-bdc6-4f7a-bb3c-fd3dfe30beff None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:43.924280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e88c-cc25-4567-99e0-44f9ce0a41b0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:43.935209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:43.941723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:43.942538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.942966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.943176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.953520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:43.954975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:43.960587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71cce9-1236-46b3-aa6f-e1057faa0e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:43.961397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71cce9-1236-46b3-aa6f-e1057faa0e14 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.961679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71cce9-1236-46b3-aa6f-e1057faa0e14 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.961955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71cce9-1236-46b3-aa6f-e1057faa0e14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.972594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71cce9-1236-46b3-aa6f-e1057faa0e14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:43.972895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:43.973699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:43.974555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:43.974555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:43.974555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:43.974555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:43.974555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 251/491] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:43.975445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1c71cce9-1236-46b3-aa6f-e1057faa0e14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:43.985155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:43.986059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:43.986866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:43.991091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 241/492] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:43.992312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d4b5b-df9b-4747-97a5-f0c8d9b4ccc3 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:43.992609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d4b5b-df9b-4747-97a5-f0c8d9b4ccc3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:43.992888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d4b5b-df9b-4747-97a5-f0c8d9b4ccc3 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:43.994209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 242/493] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:43.994891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d4b5b-df9b-4747-97a5-f0c8d9b4ccc3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.077869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.086320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa11823-43e4-486b-aeca-33f13c123012 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.103452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d4b5b-df9b-4747-97a5-f0c8d9b4ccc3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.129251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7cb79-722d-4ca4-8e73-1ee1f3eb9e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.130064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7cb79-722d-4ca4-8e73-1ee1f3eb9e1a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.130333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7cb79-722d-4ca4-8e73-1ee1f3eb9e1a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.130610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7cb79-722d-4ca4-8e73-1ee1f3eb9e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.140430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.141240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.141525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.141658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7cb79-722d-4ca4-8e73-1ee1f3eb9e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.141834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.142541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:44.142851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.143077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7cb79-722d-4ca4-8e73-1ee1f3eb9e1a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.144238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.160796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 243/494] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:44.160796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-65d7cb79-722d-4ca4-8e73-1ee1f3eb9e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:44.161647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:44.162463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:44.163289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:44.163289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:44.163289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:44.163289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:44.163289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 252/495] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:44.164273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a788f-aba0-4539-9e96-400ce9d7714e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.164582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a788f-aba0-4539-9e96-400ce9d7714e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.164862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a788f-aba0-4539-9e96-400ce9d7714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.182890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b70a788f-aba0-4539-9e96-400ce9d7714e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:44.184111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b70a788f-aba0-4539-9e96-400ce9d7714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:44.196127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.263406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 253/496] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:44.263406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a788f-aba0-4539-9e96-400ce9d7714e None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.277266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c96cd-4cce-404c-8063-3ae2aa27f69e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.312878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.313869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.314160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.314440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.315204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:44.315549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.317333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.325595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 244/497] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:44.326380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.347191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.372672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a788f-aba0-4539-9e96-400ce9d7714e None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.407447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.408360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.408648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.408924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.417136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.418642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.423193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.433734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7b49-004a-4f27-a37b-9c89709ec5a9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:44.436504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:44.437445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:44.438324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 254/498] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:44.438967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.470387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.471170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.471474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.471846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.472582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:44.472890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.474275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.488396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 245/499] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:44.489586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.562881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef8ea-ea20-4520-8455-db061eb677d4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.590556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f027e0-5fc7-4a06-8667-4b45af53a863 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.595573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.596381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.596662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.596933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.599893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.608206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:44.615953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:44.616838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 255/500] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:44.622890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.626687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.627524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.627809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.628083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.638297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.639782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:44.657004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:44.657989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:44.658949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:44.658949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:44.658949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:44.658949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:44.658949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 246/501] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:44.659321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.662888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.724073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91b2a7-93ef-4aeb-bcd5-a9af72f75fd1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.756809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5a0cd-c192-4a8c-b805-31701bc045ed None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.760232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7ba8-fc3e-43be-ac8a-32bf7bd22cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.760917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7ba8-fc3e-43be-ac8a-32bf7bd22cc5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.761196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7ba8-fc3e-43be-ac8a-32bf7bd22cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.761464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7ba8-fc3e-43be-ac8a-32bf7bd22cc5 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.762308 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7ba8-fc3e-43be-ac8a-32bf7bd22cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:44.762619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7ba8-fc3e-43be-ac8a-32bf7bd22cc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.764099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7ba8-fc3e-43be-ac8a-32bf7bd22cc5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.791982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.792791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.793072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.793349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.800096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 256/502] 10.222.0.87 () {56 vars in 1137 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:44.800543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.801395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c3d69-3589-4254-90be-b282c283945c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.801711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c3d69-3589-4254-90be-b282c283945c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.801969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c3d69-3589-4254-90be-b282c283945c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.802060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.803330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 257/503] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:44.803896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c3d69-3589-4254-90be-b282c283945c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:44.823044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:44.824193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:38:44.825386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:44.827542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 247/504] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:44.828563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.862530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.890195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:44.906599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c3d69-3589-4254-90be-b282c283945c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.929594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9fd0-c41c-437d-baf5-34a10562ecaf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:44.944254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.944930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.945218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.945492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.946222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:44.946532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.947979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.956102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 258/505] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:44.956793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:44.964962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726df7b-7670-4acb-8d9a-d5dfcfe460b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:44.965633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726df7b-7670-4acb-8d9a-d5dfcfe460b4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.965914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726df7b-7670-4acb-8d9a-d5dfcfe460b4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.966187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726df7b-7670-4acb-8d9a-d5dfcfe460b4 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.966923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726df7b-7670-4acb-8d9a-d5dfcfe460b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:44.967229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726df7b-7670-4acb-8d9a-d5dfcfe460b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:44.968640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726df7b-7670-4acb-8d9a-d5dfcfe460b4 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:44.980104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 248/506] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:44.981283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04820d78-bdd4-4c5f-a23a-6cc82bf1bb26 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:44.981571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04820d78-bdd4-4c5f-a23a-6cc82bf1bb26 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:44.981849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04820d78-bdd4-4c5f-a23a-6cc82bf1bb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:44.997594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-04820d78-bdd4-4c5f-a23a-6cc82bf1bb26 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:44.998846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-04820d78-bdd4-4c5f-a23a-6cc82bf1bb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:45.060082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f494d-734f-44b0-9f8e-acca69e60bc4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.062758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 249/507] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:45.063610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04820d78-bdd4-4c5f-a23a-6cc82bf1bb26 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.083835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.103336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.104067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.104346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.104622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.121529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.169337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04820d78-bdd4-4c5f-a23a-6cc82bf1bb26 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.177271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.178744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.206080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.206758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.207067 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.207345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.208212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:45.208523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.209949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.246214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 250/508] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:45.247023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.283626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 259/509] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:45.284372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.303272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.347691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd6168-2bcf-479f-978a-2ebcf47f4a6e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.360853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.381922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519f4e0-68d3-4b1e-935c-ab84cc4c80f6 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.382517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.383326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.383638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.383916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.393898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.395363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:45.412976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:45.413866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:45.414777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:45.414777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:45.414777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:45.414777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:45.414777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 251/510] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:45.415034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.420237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.420994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.421283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.421558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] PATH_INFO: `/v3/users/cinder` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.429101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.430569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:45.445426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:45.446511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 260/511] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:45.447829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.526147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.526538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b38484-023a-49a6-9628-429ca89619e5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.549140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7c9c-c2d1-4050-888f-c80a63418cda None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.561040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.561853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.562142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.562417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.573687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.575172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.582997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.584058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.584502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.584764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:45.592312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:45.593156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:45.593990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:45.593990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:45.593990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:45.593990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:45.593990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 252/512] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:45.594258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.612177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.653716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.655204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.693225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482099-e519-4031-9b05-80c6b0f8ac28 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.725534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.726339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.726625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.726902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.728463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.738327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.739843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.753576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 261/513] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:45.754393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:45.757989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:45.758976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:45.760200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:45.760200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:45.760200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:45.760200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:45.760200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 253/514] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:45.760200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.825050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:45.855738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42641eb1-8154-4741-96a8-9faed8426526 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.858408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae808de-0a38-4b76-9846-b9a39b3956ef None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:45.889004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.889670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.889956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.890229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.890950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:45.891255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.892661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.893900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:45.894576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:45.894869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:45.895153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:45.896035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:45.896351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:45.897787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:45.904205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 262/515] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:45.904993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.915755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 254/516] 10.222.0.87 () {56 vars in 1137 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:45.916976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:45.943095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.009137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55927e-3c77-4540-9a05-41c09634caae None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.015612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26353e04-a186-453e-8a5d-772c5e5816d2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.041272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.045082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.045763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.046056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.046330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.047063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:46.047402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.048848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.050161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.050969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.051255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.051565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.060534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 263/517] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:46.061321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.066177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.067666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:46.085717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:46.086546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:46.087727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:46.087727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:46.087727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:46.087727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:46.087727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 255/518] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:46.088258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.158296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.164907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea9a37-d95d-4c4c-9678-7ab7463d4dda None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.192706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e079c-9cd1-4498-8afe-b6a3ad55c320 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.200382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.201056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.201421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.201716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.202435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:46.202746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.204170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.215105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 264/519] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:46.215943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.226989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.227800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.228161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.229035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.236321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.237772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:46.252514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:46.253568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 256/520] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:46.254454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.293628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.319357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6eb33-2a64-479f-9f86-0dc9da0308bc None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.352542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52542d11-0ede-43ab-8f7b-7c7afe57ee91 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.352818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.353648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.353831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.354117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.378882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.386882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.387587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.387868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.388141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.388850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.390285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.410341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 257/521] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:38:46 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:38:46.411134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.424072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.425657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.522495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.533386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c94837-7221-4498-854b-73fd51f64041 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.543411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 265/522] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:46.549194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.573390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.573710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.574273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.575026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:46.575337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.576745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.591957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 258/523] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:46.592865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.610547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.656161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128bf4a-5cc2-4a5b-b650-7e95c95e0869 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.692481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.693324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.693615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.693895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.700410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7723-10a8-4f2b-b849-8008ee980d92 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.700707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.702172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:46.717617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:46.718498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:46.719332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 266/524] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:46.719843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.743531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.744470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.744760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.745370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.746499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:46.746820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.749032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.775429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 259/525] 10.222.0.87 () {56 vars in 1139 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:46.775429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.795542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.827126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9c1b1-6bf4-4a7a-b7f3-476ceddc6208 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.843531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:46.862756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.863628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.863893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.864173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.875058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.876584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.888232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1433b7-f7b5-47b5-9f1b-575abe7c5ccc None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:46.894697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:46.895582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:46.896407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:46.896407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:46.896407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:46.896407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:46.899687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 267/526] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:46.901340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:46.928275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:46.929767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:46.930234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:46.930517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:46.938314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:46.939795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:46.955021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:46.956184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:46.957214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:46.960891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 260/527] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:46.961696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.012582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1820f5-e5cf-4d4e-b93b-24d80e7622c8 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.033431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.049063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.049819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.050182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.050539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.051412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:47.051792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.053309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.065313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8edf0c-23a3-4aa8-bcaa-11d514ef4c80 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.071497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 268/528] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:47.072412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.102608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.107347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.108052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.108349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.108613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.183022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.184518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.185489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80cdf-b79c-468c-89c8-0fafad44c8f6 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.222445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.223123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.223431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.223710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.224441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:47.224749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.226143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.237896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 269/529] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:47.238723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.259152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.291467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 261/530] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:47.292301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.346638 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.359226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bd9a1-c070-47b2-b8fa-951941420fc4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.393031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.393705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.393989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.394270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.395063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:47.395412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.396858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.398369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985df2fc-0ed2-4282-afc4-93473adf60e8 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.408826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 270/531] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:47.409519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.439650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.440688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.441197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.441652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.442565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=f733becb24b14cd5842de72b62d051b9) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:47.442877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.444456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.455464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 262/532] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/role_assignments?user.id=f733becb24b14cd5842de72b62d051b9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:47.455464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.498990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.528144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e44dd-6f0d-4cad-9ab4-bc951d65daf2 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.574569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.575437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.575727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.576005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.583874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.585370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.595584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841241d-ea36-47e2-87cc-5e46196badb5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:47.600779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:47.601655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:47.602468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:47.607222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 271/533] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:47.608139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.621250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.644028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.644850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.645140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.645421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.659310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.661645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:47.682779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:47.683724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:47.684536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:47.684536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:47.684536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:47.684536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:47.684536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 263/534] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:47.685353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.734053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98cc15-1ab4-42ee-9c6d-f2e4bdd15c59 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.781459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.782122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.783649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.784117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.784718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.787214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=abaa59b4e88642c681e9563295f6fad9) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:47.788023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.790159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.796909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caec60-26bd-41cb-89cc-73ed6f91e682 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:47.799006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 272/535] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/role_assignments?user.id=abaa59b4e88642c681e9563295f6fad9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:47.800897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.852977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:47.853816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:47.854175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:47.854522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:47.855483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:47.855870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:47.857455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:47.889044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:47.900544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 264/536] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:47.911404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:47.953044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4789b-ce07-4463-b024-a8cf487cae8d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:48.025279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:48.025979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.026264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.026538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.027287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:48.027626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:48.029045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:48.050330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 273/537] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:48.057663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:48.083357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e052f6e-38ab-46ad-b030-e982a0231528 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:48.136376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:48.136913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.137210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.137491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.138348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=1315c22b7db84afd9ef41bf853b9819b) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:48.138673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:48.140214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:48.152406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 265/538] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:47 2022] GET /identity/v3/role_assignments?user.id=1315c22b7db84afd9ef41bf853b9819b&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:48.183992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d03e3-9226-44e9-9a4d-f713556bfb07 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:48.219249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:48.225569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:48.226689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.227180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.227677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.236094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:48.237907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:48.253120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:48.254014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:48.254841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:48.257431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 274/539] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:48.263096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:48.288708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:48.413301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bbbfa-8f56-47e3-8aa8-90d8c8fa029c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:48.465463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c373-7894-4069-91ff-3c3c1299ceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:48.466144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c373-7894-4069-91ff-3c3c1299ceea None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.466431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c373-7894-4069-91ff-3c3c1299ceea None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.466709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c373-7894-4069-91ff-3c3c1299ceea None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.467541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929c373-7894-4069-91ff-3c3c1299ceea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:48.467861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929c373-7894-4069-91ff-3c3c1299ceea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:48.469298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929c373-7894-4069-91ff-3c3c1299ceea None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:48.485317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 266/540] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:48.491171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:48.617271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df1817-3243-4d18-9cbf-4972ccd127e8 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:48.655394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed648-9310-4a88-8efc-399d0f3267b5 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.655394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed648-9310-4a88-8efc-399d0f3267b5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.655394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed648-9310-4a88-8efc-399d0f3267b5 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.655394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 267/541] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:48.659201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addb8a9-56c7-4fa5-ae21-15a580b1b24b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.659524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addb8a9-56c7-4fa5-ae21-15a580b1b24b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.659802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addb8a9-56c7-4fa5-ae21-15a580b1b24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.660079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:48.660974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.661429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.661886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.662902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=40f6ae0e5e164461b4c51b829be9a7d6) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:48.663472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:48.665096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:48.672496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 275/542] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:48 2022] GET /identity/v3/role_assignments?user.id=40f6ae0e5e164461b4c51b829be9a7d6&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:48.687679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8addb8a9-56c7-4fa5-ae21-15a580b1b24b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:48.689258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8addb8a9-56c7-4fa5-ae21-15a580b1b24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:48.781004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 268/543] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:38:48.817353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:48.885624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:48.931101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416e97b-0d43-4215-a381-de3dc7ffae36 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:48.970928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:48.971788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:48.972080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:48.972350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:48.984131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:48.992219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:48.993141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:48.994075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:48.995359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 276/544] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:49.001865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addb8a9-56c7-4fa5-ae21-15a580b1b24b None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:49.004165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:49.140022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addb8a9-56c7-4fa5-ae21-15a580b1b24b None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:49.197857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:49.198208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:49.198502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:49.198776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:49.199560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:49.199868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:49.201247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:49.215282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 269/545] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:49.222039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:49.336990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead6da1-f756-4717-9b5a-a06fee2292ce None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:49.353948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:49.384077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:49.386066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:49.386550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:49.387044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:49.402014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:49.407466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:49.427136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:49.428057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:49.428869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:49.428869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:49.428869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:49.428869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:49.431144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 277/546] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:49.437545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:49.524425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:49.583060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550526-09d8-4e40-899d-2d67502f98b4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:49.632942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecfb33-002c-4c37-a881-b28e607fe621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:49.633617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecfb33-002c-4c37-a881-b28e607fe621 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:49.633901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecfb33-002c-4c37-a881-b28e607fe621 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:49.634172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecfb33-002c-4c37-a881-b28e607fe621 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:49.635483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecfb33-002c-4c37-a881-b28e607fe621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:49.635797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecfb33-002c-4c37-a881-b28e607fe621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:49.637175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecfb33-002c-4c37-a881-b28e607fe621 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:49.649350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 270/547] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:49.655585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:49.791207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b62bbf-c55d-455a-8515-9d3c526a1d2c None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:49.845638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a56243-453a-42b1-96e6-607618f40144 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:49.845967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a56243-453a-42b1-96e6-607618f40144 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:49.846188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a56243-453a-42b1-96e6-607618f40144 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:49.847871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 271/548] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:49.848561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:49.849371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:49.849655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:49.849924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:49.854582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b217660-20f4-4b0e-8c0f-7a54fc8b3b82 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:49.854895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b217660-20f4-4b0e-8c0f-7a54fc8b3b82 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:49.855134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b217660-20f4-4b0e-8c0f-7a54fc8b3b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:49.872284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:49.873790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:49.881305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9b217660-20f4-4b0e-8c0f-7a54fc8b3b82 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:49.882490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9b217660-20f4-4b0e-8c0f-7a54fc8b3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:49.904732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:49.906169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:38:49.907234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:49.907788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 278/549] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:49.917058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:49.974888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 272/550] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:49.987835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:50.011074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b217660-20f4-4b0e-8c0f-7a54fc8b3b82 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:50.057811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a8d5f-53d9-4e4e-a8ce-a17f27757f79 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:50.091933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:50.092612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.092893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.093164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.094004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:50.094310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:50.095782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:50.114531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 279/551] 10.222.0.87 () {56 vars in 1135 bytes} [Fri Jun 17 12:38:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:50.120766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:50.137568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:50.211762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:50.222004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b217660-20f4-4b0e-8c0f-7a54fc8b3b82 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:50.224296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e3a9d-2d5b-4f66-baae-7ea465972277 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:50.272369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:50.274269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.276027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.277077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.304617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:50.305445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.305729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.305992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.330482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:50.338864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:50.339970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:50.340775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:50.352571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 273/552] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:50.364052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:50.365429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:50.366093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:50.455319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:50.473055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 280/553] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:50.480528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:50.601159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206e8bd-05e1-4308-8286-4ebe1bbdebc3 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:50.622813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:50.648381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:50.648600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54dd16-85f7-42e5-ace5-6c12dbd4fabd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:50.649095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.649377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.649651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.650540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:50.651656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:50.653094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:50.664737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 274/554] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:50.676481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:50.700177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a885ad-8c50-40f9-8c81-bec830a910e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:50.701006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a885ad-8c50-40f9-8c81-bec830a910e7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.701288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a885ad-8c50-40f9-8c81-bec830a910e7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.702320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a885ad-8c50-40f9-8c81-bec830a910e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.724534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a885ad-8c50-40f9-8c81-bec830a910e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:50.726779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a885ad-8c50-40f9-8c81-bec830a910e7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-55a885ad-8c50-40f9-8c81-bec830a910e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:50.754374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:50.755321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:50.756217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:50.756217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:50.756217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:50.756217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:50.756217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 281/555] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:50.758023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef349d-c0ae-4b56-8a63-74c07b4f5a5c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.758418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef349d-c0ae-4b56-8a63-74c07b4f5a5c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.758695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef349d-c0ae-4b56-8a63-74c07b4f5a5c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.761636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 282/556] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:50.762350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef349d-c0ae-4b56-8a63-74c07b4f5a5c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:50.826008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:50.828151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08192df4-5f45-41d8-a4fb-d21a47a359b1 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:50.870856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cabf7e-10e1-48e3-94ef-1f6af099a40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:50.871705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabf7e-10e1-48e3-94ef-1f6af099a40e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.871986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabf7e-10e1-48e3-94ef-1f6af099a40e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.872259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabf7e-10e1-48e3-94ef-1f6af099a40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.885693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabf7e-10e1-48e3-94ef-1f6af099a40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:50.887354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabf7e-10e1-48e3-94ef-1f6af099a40e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:50.896554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef349d-c0ae-4b56-8a63-74c07b4f5a5c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-35cabf7e-10e1-48e3-94ef-1f6af099a40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:50.906320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:50.908002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 275/557] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5ec04-9f57-4a54-9a8d-37da34b73c21 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5ec04-9f57-4a54-9a8d-37da34b73c21 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.910402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5ec04-9f57-4a54-9a8d-37da34b73c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.933215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-6fb5ec04-9f57-4a54-9a8d-37da34b73c21 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:50.935161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6fb5ec04-9f57-4a54-9a8d-37da34b73c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:50.945458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:50.947075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:50.948156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:50.949106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:50.949982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:50.950813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:50.952666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:50.965863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:50.973706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 283/558] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:50.974794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.021274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 276/559] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:51.022135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5ec04-9f57-4a54-9a8d-37da34b73c21 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.092196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.097181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45336-c90c-4540-b490-94a25d5d0706 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.127772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5ec04-9f57-4a54-9a8d-37da34b73c21 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.138939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac1674-f18e-4542-9eed-161a151596f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.139651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac1674-f18e-4542-9eed-161a151596f1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.139931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac1674-f18e-4542-9eed-161a151596f1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.140200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac1674-f18e-4542-9eed-161a151596f1 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.140931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac1674-f18e-4542-9eed-161a151596f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:51.141241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac1674-f18e-4542-9eed-161a151596f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:51.142618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac1674-f18e-4542-9eed-161a151596f1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:51.154885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 284/560] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:51.156166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f825d-eeb5-4691-bff0-208e597b7252 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.156454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f825d-eeb5-4691-bff0-208e597b7252 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.156731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f825d-eeb5-4691-bff0-208e597b7252 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.162113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 285/561] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:51.162113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f825d-eeb5-4691-bff0-208e597b7252 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.170859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.171717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.172002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.172277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.180287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:51.181829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:51.196100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:51.197021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 277/562] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:51.198491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.219804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.273657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f825d-eeb5-4691-bff0-208e597b7252 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.308075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398d161-5b22-4a4d-a817-9345fc7034aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.308906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398d161-5b22-4a4d-a817-9345fc7034aa None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.309188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398d161-5b22-4a4d-a817-9345fc7034aa None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.309463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398d161-5b22-4a4d-a817-9345fc7034aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.315045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61f9e1-1793-44fb-8797-68548cf5b079 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.323068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398d161-5b22-4a4d-a817-9345fc7034aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f398d161-5b22-4a4d-a817-9345fc7034aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:51.331089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:51.332519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:51.333370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 286/563] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:51.334077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e984cd-decc-42d4-975f-ee38d63c2d0a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.334077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e984cd-decc-42d4-975f-ee38d63c2d0a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.334727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e984cd-decc-42d4-975f-ee38d63c2d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.351889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-42e984cd-decc-42d4-975f-ee38d63c2d0a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:51.353055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-42e984cd-decc-42d4-975f-ee38d63c2d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:51.358158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.358968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.359254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.359559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] PATH_INFO: `/v3/users/glance` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.366519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.370618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:51.371438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:51.389500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:51.390646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 278/564] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:51.391940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.422446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 287/565] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:51.423309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e984cd-decc-42d4-975f-ee38d63c2d0a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.492780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcedbb-d777-4e45-96d6-676050b3b360 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.522624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e984cd-decc-42d4-975f-ee38d63c2d0a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.525549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550b049-0ffa-45b8-ad2f-432fe16698ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.526211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b049-0ffa-45b8-ad2f-432fe16698ff None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.526494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b049-0ffa-45b8-ad2f-432fe16698ff None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.526770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b049-0ffa-45b8-ad2f-432fe16698ff None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.527591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550b049-0ffa-45b8-ad2f-432fe16698ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:51.527890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550b049-0ffa-45b8-ad2f-432fe16698ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:51.529323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550b049-0ffa-45b8-ad2f-432fe16698ff None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:51.541472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 279/566] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:51.542756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6285-4e62-455a-aa86-7cff8b004e41 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.543040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6285-4e62-455a-aa86-7cff8b004e41 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.543313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6285-4e62-455a-aa86-7cff8b004e41 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.547431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 280/567] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:51.547431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb6285-4e62-455a-aa86-7cff8b004e41 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.561989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.562671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.562952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.563226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.564013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:51.564318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:51.565694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:51.574226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 288/568] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:51.574990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.590680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.648506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb6285-4e62-455a-aa86-7cff8b004e41 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.676222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138fb8-0b75-4cff-8059-26d3ba437eca None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.682641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.683311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.683615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.683885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.684704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:51.685005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:51.686422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:51.696721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.707358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 281/569] 10.222.0.87 () {56 vars in 1137 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:51.708169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.712152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff70b3-9f5b-4c64-b8cc-623c38602674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.713529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff70b3-9f5b-4c64-b8cc-623c38602674 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.713816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff70b3-9f5b-4c64-b8cc-623c38602674 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.714092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff70b3-9f5b-4c64-b8cc-623c38602674 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.724599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff70b3-9f5b-4c64-b8cc-623c38602674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-21ff70b3-9f5b-4c64-b8cc-623c38602674 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:51.732302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:51.733198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:51.734003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:51.738445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 289/570] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:51.739698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324e70f-beec-458b-8ad8-b3f3d0f3fb77 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.739987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324e70f-beec-458b-8ad8-b3f3d0f3fb77 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.740270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324e70f-beec-458b-8ad8-b3f3d0f3fb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.757106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c324e70f-beec-458b-8ad8-b3f3d0f3fb77 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:51.758261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c324e70f-beec-458b-8ad8-b3f3d0f3fb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:51.811791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffef0a-2bb1-49ff-bb50-acb6cbcc7b77 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.821446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.827167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 290/571] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:51.828025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324e70f-beec-458b-8ad8-b3f3d0f3fb77 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.852140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.852806 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.853089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.853390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:51.854316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=1a31e162778e4824b7bafd5dcd9b4b4b) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:51.854625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:51.856116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:51.861671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 282/572] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/role_assignments?user.id=1a31e162778e4824b7bafd5dcd9b4b4b&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:51.862451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:51.931714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:51.951969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324e70f-beec-458b-8ad8-b3f3d0f3fb77 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.977953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e57b65-fb64-4b77-a68d-72b3bc603c52 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:51.989228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:51.990056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:51.990350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:51.990641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.002156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.003681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.016988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.017833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.018108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.018381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:52.023204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:52.024227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:52.025172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:52.025172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:52.025172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.025172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:52.029459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 291/573] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:52.030662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.030767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.032051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:52.052351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:52.053388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:52.054309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:52.054309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:52.054309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.054309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:52.054309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 283/574] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:52.055491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.090504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:52.149815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac280ea4-2434-4b80-8ce3-96789a4e7d41 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.162838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b241d-9000-4356-8cf7-c71703879cb7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.176654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:52.189097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.190069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.190519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.190978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.191919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:52.192384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.194183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.198429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.199248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.199565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.199845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.204158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 292/575] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:52.205108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.215555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:52.223494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:52.224414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:52.225270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 284/576] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:52.225941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.318474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba1bba-0ec6-4de0-bbe0-ed7d690f7e3a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.330679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8af71-7d93-4e01-9b69-44d45c9b8f17 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.334676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:52.356377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.357057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.357330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.357612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.358351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:52.358661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.360093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.367644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.368495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.369934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.370356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.371275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:52.371791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.372285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 293/577] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:52.373104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.373760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.387956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 285/578] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:52.389029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.421629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:52.488633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74960830-2c5e-46e5-b896-3c1ab2f7d416 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.493152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e573-ffbc-44cf-80dc-24c713a2b883 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.527605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.528098 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.528292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.528554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.528826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.528921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.529180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.529456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.529633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:52.529871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.531234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.544498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 286/579] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:52.544498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.545977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.547473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:52.566176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:52.567126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:52.568076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:52.568076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:52.568076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.568076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:52.568076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 294/580] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:52.568823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.596157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:52.646460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:52.654537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40129-bec3-48a5-a157-1c9484b66804 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.673710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074deb59-f0cf-4fc1-b5df-5e33e6c9a0ce None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.689978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.690778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.691058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.691333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.698982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.700378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.711293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.712144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.712433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.712804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:52.714805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:52.715981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 287/581] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:52.716963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.726174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.727689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:52.741765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:52.742645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 295/582] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:52.744130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.830871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44685-bb71-4828-bfca-fbf91fe64fb0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.833020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:52.848023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca4881-3ecf-4ed0-8c16-7e8ac720f267 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:52.867976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.869255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.869782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.870293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.881296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:52.881958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:52.882237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:52.882508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:52.882623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.883226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:52.883558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:52.884794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.885126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:52.897507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 296/583] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:52.898291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:52.903480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:52.904323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:52.905201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:52.905201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:52.905201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:52.905201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:52.907094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 288/584] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:52.908111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:52.919192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.014513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565be9b2-d5d0-4a91-b7a7-6cb2d6079079 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.018381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2f7f-f83d-4037-b116-3015f8597bae None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.051861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.052551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.052834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.053105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.053959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:53.054269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.054466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.055219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.055648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.055761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.056082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.057000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:53.057439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.058936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.071436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 289/585] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:53.072460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.083149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 297/586] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:38:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:53.083149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.087065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.148488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.186816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a8d7b-4661-4c6c-9121-b2f9d5d239ff None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.196507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c5e9-d8bf-4169-b8e2-c58be719bac0 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.220309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.221264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.221681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.221989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.229393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.230921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.231025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.231722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.232007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.232287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.233026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:53.233684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.234690 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:53.245610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:53.256571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:38:53.257549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:53.258095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 298/587] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:53.258169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 290/588] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:53.258235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.258307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.318058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.357281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729ebb-c91d-48a9-9ee5-d8508d0f2eae None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.361126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c62642-1f31-4058-ad1c-d9aa03610384 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.384861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.394546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.395349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.395657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.395986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.399206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.400167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.400658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.400951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/f733becb24b14cd5842de72b62d051b9/roles/e381464c740146119ff513d42bb0044b` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.406445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.408050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.416939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=f733becb24b14cd5842de72b62d051b9, role_id=e381464c740146119ff513d42bb0044b)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.418499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:53.427867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:53.428819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:53.430429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:53.430429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:53.430429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:53.430429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:53.430429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 291/589] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:53.430429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.437924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 299/590] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:53 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/f733becb24b14cd5842de72b62d051b9/roles/e381464c740146119ff513d42bb0044b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:53.438753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.555531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f57fcf-9ca2-4847-9776-85342f3aea08 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.589053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a422f3-222d-40b2-8ed0-de09477983fd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.595135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.596429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.596925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.597421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.605831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.606701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.609238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:53.625725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:53.626701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 12:38:53.627631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:53.630245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 292/591] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:53.631321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.633175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.633939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.634294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.634765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.635776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:53.636361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.637952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.663431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 300/592] 10.222.0.87 () {56 vars in 1137 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:53.663431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.679542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.746084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eb6ff-0ecc-44a1-88a9-1b33004c01f1 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.792164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.793261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.794014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.794388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.795278 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:53.795705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.797748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.809049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4351-d57a-4db6-9466-44074c825a09 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.814764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 293/593] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:53.816064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.850882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8738050-93ae-4339-956f-4b300970544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:53.851694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8738050-93ae-4339-956f-4b300970544e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:53.852080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8738050-93ae-4339-956f-4b300970544e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:53.852456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8738050-93ae-4339-956f-4b300970544e None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:53.853593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8738050-93ae-4339-956f-4b300970544e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:53.854300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8738050-93ae-4339-956f-4b300970544e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:53.856247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8738050-93ae-4339-956f-4b300970544e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:53.875133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 301/594] 10.222.0.87 () {56 vars in 1139 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:53.876167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8738050-93ae-4339-956f-4b300970544e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:53.907405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.952317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:53.960101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cd305-63c0-4b1d-af34-49c9c11d7df6 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:53.991974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8738050-93ae-4339-956f-4b300970544e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.010012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.010911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.011192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.011493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.022448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.023953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.033418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f55827-3e41-4a63-a2b6-703f1493f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.034749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f55827-3e41-4a63-a2b6-703f1493f042 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.035251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f55827-3e41-4a63-a2b6-703f1493f042 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.035778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f55827-3e41-4a63-a2b6-703f1493f042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:54.042433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:54.043334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:54.044293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:54.044293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:54.044293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:54.044293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:54.045274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f55827-3e41-4a63-a2b6-703f1493f042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.046763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f55827-3e41-4a63-a2b6-703f1493f042 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.047260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 294/595] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:54.048356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-e0f55827-3e41-4a63-a2b6-703f1493f042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:54.063021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:54.063983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:54.064845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 302/596] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:54.065901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a04f03-fc9f-43db-beb1-2934b1d78ea3 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.066008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a04f03-fc9f-43db-beb1-2934b1d78ea3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.066217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a04f03-fc9f-43db-beb1-2934b1d78ea3 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.071754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 303/597] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:54.072475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a04f03-fc9f-43db-beb1-2934b1d78ea3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.172877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8e405-c335-47f7-8d82-03ac2d51429c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.178709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:54.190341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a04f03-fc9f-43db-beb1-2934b1d78ea3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.208497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.209310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.209596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.209876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.221084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.222552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.228523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e4719-b86d-470f-905e-a950d26448ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.229212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e4719-b86d-470f-905e-a950d26448ca None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.229474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e4719-b86d-470f-905e-a950d26448ca None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.229748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e4719-b86d-470f-905e-a950d26448ca None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.230477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e4719-b86d-470f-905e-a950d26448ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:54.230790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e4719-b86d-470f-905e-a950d26448ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.232258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e4719-b86d-470f-905e-a950d26448ca None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:54.240370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:54.241245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:54.242112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:54.242112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:54.242112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:54.242112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:54.246570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 304/598] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:54.246738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 295/599] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:54.247430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.249140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf3b2d-5c63-42df-a8f8-150dc6829bfc None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.249500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf3b2d-5c63-42df-a8f8-150dc6829bfc None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.249868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf3b2d-5c63-42df-a8f8-150dc6829bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.252718 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:54.269294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b1cf3b2d-5c63-42df-a8f8-150dc6829bfc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:54.270505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b1cf3b2d-5c63-42df-a8f8-150dc6829bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:54.341727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 305/600] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:54.342596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf3b2d-5c63-42df-a8f8-150dc6829bfc None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.365464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80f9e-630c-4c65-a40a-c0695b19b1a7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.401598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.402683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.402974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.403545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.404480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:54.404808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.406850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.426569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 296/601] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:54.427452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.431055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:54.451549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf3b2d-5c63-42df-a8f8-150dc6829bfc None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.489645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.490319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.490607 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.490888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.491660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:54.491972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.492810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:54.493355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.504883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 306/602] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:54.505751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.545488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5030fd7-1924-4ecf-aba4-4480fd9acf76 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.580638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.581455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.581743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.582024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.589839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.591325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:54.606570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:54.607790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:54.608750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 297/603] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:54.609244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.620485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115acc79-e32c-4cc0-abb0-a0b1d4e2af90 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.658859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.659711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.659993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.660266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.668473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:54.671177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:54.680333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:54.681556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 307/604] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:54.682716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.736156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092976-efdb-4c1f-9ac7-405127a6fba7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.760836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:54.773307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.774216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.774652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.774772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/abaa59b4e88642c681e9563295f6fad9/roles/e381464c740146119ff513d42bb0044b` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.790362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=abaa59b4e88642c681e9563295f6fad9, role_id=e381464c740146119ff513d42bb0044b)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.791300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee21b51-3998-4e2a-a117-00fd9b9beeba None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.792391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.806122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 298/605] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:54 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/abaa59b4e88642c681e9563295f6fad9/roles/e381464c740146119ff513d42bb0044b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:54.806933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.831648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:54.832221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:54.832509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:54.832839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:54.842208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:54.843339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:54.859935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:54.860909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:54.861785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 308/606] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:54.862787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:54.932200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:54.954253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d6957-d8d2-48ff-8da1-4a8f999283cf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:54.983148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e3cf2-f396-469a-a908-7bccf43af82a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:55.012439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.013498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.014187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.014715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.016350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:55.016809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.019297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.023327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.024155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.024541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.024917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.025857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:55.026266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.027806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.035793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 299/607] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:55.036905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.037797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 309/608] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:55.044349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.103870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:55.157307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf823-f43d-4bb4-84b9-9ce6f1669c51 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:55.174887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30471d1b-cdf2-4ec9-91c0-defb0ce1ff35 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:55.183167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:55.196373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.197437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.197974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.198151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/1315c22b7db84afd9ef41bf853b9819b/roles/e381464c740146119ff513d42bb0044b` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.214165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=1315c22b7db84afd9ef41bf853b9819b, role_id=e381464c740146119ff513d42bb0044b)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.215762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.222616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.223329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.223643 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.223928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.224743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:55.225047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.226551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.239548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 310/609] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:55 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/1315c22b7db84afd9ef41bf853b9819b/roles/e381464c740146119ff513d42bb0044b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:55.240475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 300/610] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:55.241119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.251424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.434209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:55.442601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056cb99-4edf-4e32-9ac9-ccb53e1d950f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:55.445940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d146f-aad4-4edd-b33d-ee57600cf273 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:55.490037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a8be3-cb90-4ce8-8a01-e1c63ba4eabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.491155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a8be3-cb90-4ce8-8a01-e1c63ba4eabb None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.491518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a8be3-cb90-4ce8-8a01-e1c63ba4eabb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.491884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a8be3-cb90-4ce8-8a01-e1c63ba4eabb None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/40f6ae0e5e164461b4c51b829be9a7d6/roles/e381464c740146119ff513d42bb0044b` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.493484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.494718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.495071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.495437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.506952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.508755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.511269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a8be3-cb90-4ce8-8a01-e1c63ba4eabb None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=40f6ae0e5e164461b4c51b829be9a7d6, role_id=e381464c740146119ff513d42bb0044b)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.513385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a8be3-cb90-4ce8-8a01-e1c63ba4eabb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:55.529888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:55.531200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:55.532593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:55.532593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:55.532593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:55.532593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:55.533007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 311/611] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:55.539436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.539617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 301/612] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:55 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/40f6ae0e5e164461b4c51b829be9a7d6/roles/e381464c740146119ff513d42bb0044b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:38:55.578554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:55.688552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2fe40-1747-4c82-b56f-568e52f41852 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:55.696673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8b39f-8a3e-4df7-b964-b34c56660015 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.696954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8b39f-8a3e-4df7-b964-b34c56660015 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.697277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8b39f-8a3e-4df7-b964-b34c56660015 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.698954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 302/613] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:38:55.704592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a82c-d647-4447-b538-17b9fe140fbd None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.705190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a82c-d647-4447-b538-17b9fe140fbd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.705673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a82c-d647-4447-b538-17b9fe140fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.729196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.730083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.730446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.730796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.731673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:55.732056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.733556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.747655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 312/614] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:55.752458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0766a82c-d647-4447-b538-17b9fe140fbd None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:55.754368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.757524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0766a82c-d647-4447-b538-17b9fe140fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:55.859996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 303/615] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:55.880948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:55.884201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766a82c-d647-4447-b538-17b9fe140fbd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.899641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea634e-14b2-4847-96c8-95fce570cc12 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:55.935772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:55.936625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:55.936907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:55.937179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:55.946055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:55.947629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:55.961128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:55.962390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:55.963666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:55.964754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 313/616] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:38:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:55.970216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:55.987963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766a82c-d647-4447-b538-17b9fe140fbd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.025304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.026166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.026454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.026732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.037971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:56.046392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:56.048582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:56.050540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 304/617] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:56.059401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.084593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763f092-73a7-4bd8-9829-dd2d24ed34f7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.124654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.125478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.125770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.126048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.126903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:56.127213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.128769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.153601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 314/618] 10.222.0.87 () {56 vars in 1135 bytes} [Fri Jun 17 12:38:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:56.154973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.156724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb1c80-4ad3-4492-b54c-5fa0e9b64d61 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.157833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:56.187429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:56.192749 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.193294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.193588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.193884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.194645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:56.194956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.196450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.205657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 305/619] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:56.211041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.294136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a64af-28d4-406d-92a6-ad7b77976d9c None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.334101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.335449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.336051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.336548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.348593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.350531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:56.370042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:56.370998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:56.371924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:56.371924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:56.371924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:56.371924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:56.377747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befa6a7-f2ed-4243-8dca-fd1f14b2043f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.378074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 315/620] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:56.380600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.413253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.414073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.414648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.414941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.428280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.430391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.457737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:56.470406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:56.471324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:56.473031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:56.473031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:56.473031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:56.473031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:56.475955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 306/621] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:56.484307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.536581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf410-af9d-4f7f-8927-f7b8971c93f7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.573094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:56.598647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.599886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.600991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.601631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.603646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:56.604322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.606653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.637107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 316/622] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:56.640958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.687303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc27b4-15f7-4751-9cb4-46cfb2951a62 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.729283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.729883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.730162 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.730461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.731182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:56.731516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.732909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.745997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 307/623] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:56.750757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.772629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414314e-acd0-4ee9-9563-e37a788b1362 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.788172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:56.823071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.824398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.824398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.824657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.848216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.849949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.859029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fcef6-5742-477e-b985-20b1f12a079d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:56.868266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:56.879868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:56.880880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:56.881756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:56.887825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 317/624] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:56.893914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:56.908725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:56.909552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:56.909831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:56.910102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] PATH_INFO: `/v3/users/glance` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:56.919940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:56.921396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:56.938103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:56.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:56.940022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 308/625] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:56.946244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:57.095182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83545b37-eca9-4a6f-b05e-cf49a456e39f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:57.138873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc280c65-d21e-44d1-b601-db9fee59c029 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:57.157479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:57.158191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.158475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.158755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.159610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:57.159929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:57.161450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:57.188826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 318/626] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:57.196379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:57.205133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db18f4d-5d51-42d7-880f-8067a3d24bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:57.205900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db18f4d-5d51-42d7-880f-8067a3d24bc7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.206280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db18f4d-5d51-42d7-880f-8067a3d24bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.206706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db18f4d-5d51-42d7-880f-8067a3d24bc7 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.207679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db18f4d-5d51-42d7-880f-8067a3d24bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:57.208118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db18f4d-5d51-42d7-880f-8067a3d24bc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:57.209684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db18f4d-5d51-42d7-880f-8067a3d24bc7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:57.229848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 309/627] 10.222.0.87 () {56 vars in 1137 bytes} [Fri Jun 17 12:38:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:57.231548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885393cf-32e9-4b9c-a59d-13308c5844e3 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.232013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885393cf-32e9-4b9c-a59d-13308c5844e3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.232463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885393cf-32e9-4b9c-a59d-13308c5844e3 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.233870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 310/628] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:57.238739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885393cf-32e9-4b9c-a59d-13308c5844e3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:57.249413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:57.266135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:57.344856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefded3-cfa3-4dca-a6b9-26a41c96e9c2 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:57.352655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885393cf-32e9-4b9c-a59d-13308c5844e3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:57.388584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:57.389497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.390046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.390549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.391577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f6529-d73e-4c8c-a9cf-e58811f765f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:57.392603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6529-d73e-4c8c-a9cf-e58811f765f3 None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.392958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6529-d73e-4c8c-a9cf-e58811f765f3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.393305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6529-d73e-4c8c-a9cf-e58811f765f3 None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/1a31e162778e4824b7bafd5dcd9b4b4b/roles/e381464c740146119ff513d42bb0044b` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.419243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f6529-d73e-4c8c-a9cf-e58811f765f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=1a31e162778e4824b7bafd5dcd9b4b4b, role_id=e381464c740146119ff513d42bb0044b)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:57.421488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f6529-d73e-4c8c-a9cf-e58811f765f3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:57.441945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 319/629] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:38:57 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/1a31e162778e4824b7bafd5dcd9b4b4b/roles/e381464c740146119ff513d42bb0044b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:38:57.443736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b2e6-6c64-4450-ae9f-ab442d1a0914 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.444479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b2e6-6c64-4450-ae9f-ab442d1a0914 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.444830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b2e6-6c64-4450-ae9f-ab442d1a0914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.466908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0eb1b2e6-6c64-4450-ae9f-ab442d1a0914 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:57.467867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:57.468168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0eb1b2e6-6c64-4450-ae9f-ab442d1a0914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:57.469808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:57.504385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:57.571023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 320/630] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:57.609363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 311/631] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:57.615567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b2e6-6c64-4450-ae9f-ab442d1a0914 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:57.616596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:57.643240 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:57.734985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669cc0f-e3a2-4b6b-8607-2854e4cd6172 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:57.762040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:57.774496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b2e6-6c64-4450-ae9f-ab442d1a0914 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:57.774683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:57.775309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.775834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.776273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.789154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:57.798819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:57.799818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:57.800679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 312/632] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:57.810175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:57.818596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:57.819465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:57.819753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:57.820027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:57.838732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:57.840228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:57.858920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:57.859960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:57.860846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:57.860846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:57.860846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:57.860846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:57.860846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 321/633] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:57.871395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:57.987886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:57.999922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2467d2-40bb-41aa-b1df-854278b385f2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.038184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.039019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.039411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.039788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.040667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:58.041495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:58.042955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:58.051918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 313/634] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:58.057382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.057637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1728821-486b-44e3-8202-dfabe645efe2 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.095140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.096157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.096603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.097035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.097965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:58.098431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:58.100087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:58.122508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 322/635] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:38:58.127505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.176772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:58.236684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6166e3-0065-4f58-b4a4-21df2cb0b33c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.238447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfff383-edd3-4d35-be03-fbdf8990bf6b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.268062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:58.278389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce80513-f2b9-4a62-a27d-e84b0747efed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.279659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80513-f2b9-4a62-a27d-e84b0747efed None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.280232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80513-f2b9-4a62-a27d-e84b0747efed None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.280774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80513-f2b9-4a62-a27d-e84b0747efed None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.282896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab411c-d947-46b7-af50-308513dea1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.283881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab411c-d947-46b7-af50-308513dea1bb None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.285109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab411c-d947-46b7-af50-308513dea1bb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.287786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab411c-d947-46b7-af50-308513dea1bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.289922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80513-f2b9-4a62-a27d-e84b0747efed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:58.293066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80513-f2b9-4a62-a27d-e84b0747efed None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:58.300707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab411c-d947-46b7-af50-308513dea1bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:58.302195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab411c-d947-46b7-af50-308513dea1bb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-6ce80513-f2b9-4a62-a27d-e84b0747efed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:38:58.308231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:58.309170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:38:58.310045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:58.310460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 323/636] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:58.311577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c7bd-022d-4037-9c11-a7ef751f9a59 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.311774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c7bd-022d-4037-9c11-a7ef751f9a59 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.312050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c7bd-022d-4037-9c11-a7ef751f9a59 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.313601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 324/637] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:58.314667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9d556-3d67-45e9-9eeb-99f8c329d9da None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.314951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9d556-3d67-45e9-9eeb-99f8c329d9da None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.315299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9d556-3d67-45e9-9eeb-99f8c329d9da None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.319403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 325/638] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:38:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:38:58.319403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9d556-3d67-45e9-9eeb-99f8c329d9da None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-41ab411c-d947-46b7-af50-308513dea1bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:58.328374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:58.329260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:58.330132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:58.330132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:58.330132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:58.330132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:58.330132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 314/639] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:38:58.331533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297598f1-f7ea-48ac-8bc9-7e5a18a9dff4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.331533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297598f1-f7ea-48ac-8bc9-7e5a18a9dff4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.331533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297598f1-f7ea-48ac-8bc9-7e5a18a9dff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.350365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-297598f1-f7ea-48ac-8bc9-7e5a18a9dff4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:58.351785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-297598f1-f7ea-48ac-8bc9-7e5a18a9dff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:58.421764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 315/640] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:58.422978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978cc3-1d71-482e-80a4-b9b82ac65f0c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.423275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978cc3-1d71-482e-80a4-b9b82ac65f0c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.423581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb978cc3-1d71-482e-80a4-b9b82ac65f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.440477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-bb978cc3-1d71-482e-80a4-b9b82ac65f0c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:38:58.441125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9d556-3d67-45e9-9eeb-99f8c329d9da None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.441653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-bb978cc3-1d71-482e-80a4-b9b82ac65f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:38:58.466237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:58.477131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.477813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.478095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.478369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:58.479997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:58.481897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:58.499423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 326/641] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:58.499423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.519473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 316/642] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:38:58.520534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978cc3-1d71-482e-80a4-b9b82ac65f0c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.551455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:58.608244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5165e75-458f-4c1e-8e28-7f2fbbbadd4b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.639394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb978cc3-1d71-482e-80a4-b9b82ac65f0c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.647661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd59d-245e-4f22-954a-427269499180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.648342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd59d-245e-4f22-954a-427269499180 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.648629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd59d-245e-4f22-954a-427269499180 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.648905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd59d-245e-4f22-954a-427269499180 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.649635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bd59d-245e-4f22-954a-427269499180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:58.649944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bd59d-245e-4f22-954a-427269499180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:58.651449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bd59d-245e-4f22-954a-427269499180 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:58.664112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 327/643] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:58.664921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd59d-245e-4f22-954a-427269499180 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.680267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.681104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.681457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.681786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.692063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:58.699819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:58.700819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:58.701652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 317/644] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:58.702334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.721670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:58.775777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd59d-245e-4f22-954a-427269499180 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.783548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:58.812722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9096b-3aec-4a95-aae6-c3fc0ba23b5e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:58.814304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.814993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.815274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.816428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.816428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=f733becb24b14cd5842de72b62d051b9) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:58.816723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:58.818229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:58.825829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 328/645] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/role_assignments?user.id=f733becb24b14cd5842de72b62d051b9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:58.826684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.863710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:58.864983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:58.865458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:58.865952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:58.880002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:38:58.891997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:58.893019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:38:58.894034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:58.902304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 318/646] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:58.903483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:58.984173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7377c-5ceb-4e72-8c6e-0df86339d630 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.011786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.021295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe0446-6775-401e-a34a-38b40abe3ac4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.021931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.022749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.023037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.023315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.031857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.033324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:59.048170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:59.049169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:59.050106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 329/647] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:59.050682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.062327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.063010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.063528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.063644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.064336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:59.065400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.066795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.076290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 319/648] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:59.077208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.090555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.192649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1672a-accc-4dbc-8bde-e3963abb3e84 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.200552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9ad07-3ae2-4ae2-bbe2-8f6195b0e778 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.233843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.234913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.235398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.235851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.236804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:59.237459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.239415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.240796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.241743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.242027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.242301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.243669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:59.244014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.245516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.250369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 330/649] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:59.251493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.262162 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.275520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 320/650] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:59.276424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.381862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.394183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0ea5b-640e-498d-8a60-3172fc66c363 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.397736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4b1f8-f25c-4843-80e9-3087c48395d2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.433566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.434931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.435610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.436209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.436595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.437459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.437744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.438017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.448997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.450802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.451958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.453431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:59.470340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:59.471259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:59.472149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:59.472149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:59.472149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:59.472149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:38:59.472377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:59.473132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:59.474010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:38:59.474010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:38:59.474010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:38:59.474010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:59.474010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 331/651] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:59.474010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.477263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 321/652] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:59.478269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.551779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.581371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42db17-42ac-49e8-a199-a88e59804a95 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.581894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73b5fd-dd40-4e9f-a555-8b738af29788 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.615508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.616172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.616500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.616776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.617872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.618549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.618835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.619115 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.620054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:59.620458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.621843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.632095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.639587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 322/653] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:59.641385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.691152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.692659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.751824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bb44c-6ff5-431b-9009-1a4e641d2828 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.763659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.792385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.793440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.793954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.794429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.795362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:38:59.795979 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.797561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.799092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 332/654] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:38:59.800068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.815329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 323/655] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:38:59.816806 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.856912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:38:59.911709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca6f49-d5ab-43cb-b4dd-5473c038f1be None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.930470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dbc85-005f-4d4c-9ff9-8cdc3b5dc44c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:38:59.945964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.946784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.947078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.947362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.954430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.955823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:38:59.966640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47196-0170-4fd4-8baa-e1abc5e63476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:38:59.967642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47196-0170-4fd4-8baa-e1abc5e63476 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:38:59.968088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47196-0170-4fd4-8baa-e1abc5e63476 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:38:59.968528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47196-0170-4fd4-8baa-e1abc5e63476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:38:59.970862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:38:59.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:38:59.973126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:38:59.976808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 333/656] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:38:59.977645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:38:59.982920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47196-0170-4fd4-8baa-e1abc5e63476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:38:59.984582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47196-0170-4fd4-8baa-e1abc5e63476 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-54a47196-0170-4fd4-8baa-e1abc5e63476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:00.003490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:00.004564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:00.005654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:00.005654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:00.005654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:00.005654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:00.007356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 324/657] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:00.008903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e41f-a868-42c6-a3f4-cad072896199 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.009190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e41f-a868-42c6-a3f4-cad072896199 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.009462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e41f-a868-42c6-a3f4-cad072896199 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.010931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 325/658] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:00.015920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e41f-a868-42c6-a3f4-cad072896199 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.041892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.085888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed364f7e-7bd3-435a-8f7a-1af0cbfead91 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.088504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.131115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.131326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e41f-a868-42c6-a3f4-cad072896199 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.131968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.132250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.132529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.140478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.142020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:00.157730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:00.158696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 12:39:00.159662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:00.163599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 334/659] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:38:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:00.164417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.171104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00365a7-4cdc-4a59-9b4e-fdc807c457ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.171817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00365a7-4cdc-4a59-9b4e-fdc807c457ba None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.172103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00365a7-4cdc-4a59-9b4e-fdc807c457ba None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.172381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00365a7-4cdc-4a59-9b4e-fdc807c457ba None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.173232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00365a7-4cdc-4a59-9b4e-fdc807c457ba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:00.173548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00365a7-4cdc-4a59-9b4e-fdc807c457ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.175000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00365a7-4cdc-4a59-9b4e-fdc807c457ba None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.197627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 326/660] 10.222.0.87 () {56 vars in 1137 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:00.197627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0fa28-ebfa-4b6b-a172-e0604290449d None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.197627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0fa28-ebfa-4b6b-a172-e0604290449d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.197627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0fa28-ebfa-4b6b-a172-e0604290449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.212183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-67e0fa28-ebfa-4b6b-a172-e0604290449d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:00.213493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-67e0fa28-ebfa-4b6b-a172-e0604290449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:00.276150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649be70-e7f9-430e-9e87-e9aeea97138f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.284219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 327/661] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:00.285211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0fa28-ebfa-4b6b-a172-e0604290449d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.308404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.315999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.316749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.317039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.317315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.318053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:00.318362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.319773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.335496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 335/662] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:00.335496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.362246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.401779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0fa28-ebfa-4b6b-a172-e0604290449d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.437031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.437882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.438169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.438444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.439281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:00.439620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.440216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc21b00-2f41-4c84-ac4f-4e8cce210dd7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.441129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.464835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 328/663] 10.222.0.87 () {56 vars in 1139 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:00.465418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.476062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.476776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.477052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.477320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.531736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.550802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.552363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.566165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cf71b-9ee2-4d14-bc13-c29445d89318 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.577681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.601608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.602447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.602697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.602966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.613705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:00.621405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:00.622324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:00.623191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 329/664] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:00.623935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.660006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 336/665] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:39:00.660856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.743893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b4516-8e8b-4dba-ad53-ee0a03436ff0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.765707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7878d-1bbc-4976-849c-2157dd80eb79 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.775950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.785481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.786334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.786630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.786965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.792811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:00.795158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.796737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.804798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.805458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.805739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.806015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:00.812103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:00.812990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:00.813893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 330/666] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:00.814831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:00.882781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.885283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.918778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c4f19-4b4d-445e-8f7b-03c5a375fad2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:00.952740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:00.953533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:00.953877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:00.954221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:00.954973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:00.955277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:00.956676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:00.965129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 331/667] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:00.965921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.003887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 337/668] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:39:01.008938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.018698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.035687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.084047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb891c-dddc-4524-b1fc-51bf7d733bf7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.123342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f8ae8-2a59-4c03-a04c-c1aadb45a33e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.130239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.131057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.131758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.132039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.144728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.146097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.159114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.159823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.160108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.160383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.161178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:01.161489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.162920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:01.163438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:01.164419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:01.165321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:01.165321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:01.165321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.165321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:01.165321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 332/669] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:01.165661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.180261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 338/670] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:01.180261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.261899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.268243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d3841-d374-4d00-add6-a9774e7b14a1 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.269778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.277925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d4999-eb0b-42a6-9d08-979321c68dcd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.302496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3645bb9-deca-4d4d-9bed-72f5c07e9e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.303295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3645bb9-deca-4d4d-9bed-72f5c07e9e70 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.303634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3645bb9-deca-4d4d-9bed-72f5c07e9e70 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.303919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3645bb9-deca-4d4d-9bed-72f5c07e9e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.312675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.313536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.313889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.314173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.315609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3645bb9-deca-4d4d-9bed-72f5c07e9e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.317058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3645bb9-deca-4d4d-9bed-72f5c07e9e70 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.324595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.326056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-f3645bb9-deca-4d4d-9bed-72f5c07e9e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:01.333807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:01.334836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:01.335904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:01.335904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:01.335904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.335904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:01.335904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 333/671] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:01.336562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3162c9-95db-4fa3-ac2d-98b3546653be None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.336815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3162c9-95db-4fa3-ac2d-98b3546653be None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.337095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3162c9-95db-4fa3-ac2d-98b3546653be None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.338894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 334/672] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:01.339626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3162c9-95db-4fa3-ac2d-98b3546653be None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:01.345608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:01.346459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:01.347322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:01.347322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:01.347322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.347322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:01.347322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 339/673] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:01.348937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.448460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8584-8574-496d-bf8d-5aca0dfe3135 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.452574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3162c9-95db-4fa3-ac2d-98b3546653be None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.481873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.482531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.482811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.483084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.483950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=abaa59b4e88642c681e9563295f6fad9) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:01.484269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.485716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.487356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a986-8402-4fed-967b-137d7e98fad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.488029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a986-8402-4fed-967b-137d7e98fad7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.488321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a986-8402-4fed-967b-137d7e98fad7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.488601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a986-8402-4fed-967b-137d7e98fad7 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.488891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.489346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a986-8402-4fed-967b-137d7e98fad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:01.489644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a986-8402-4fed-967b-137d7e98fad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.491045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a986-8402-4fed-967b-137d7e98fad7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.492628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 340/674] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/role_assignments?user.id=abaa59b4e88642c681e9563295f6fad9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:01.493806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.510613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 335/675] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:01.512301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4e13f-8432-4397-a31a-3f1ba0be66ce None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.512598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4e13f-8432-4397-a31a-3f1ba0be66ce None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.512860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4e13f-8432-4397-a31a-3f1ba0be66ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.533498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ecd4e13f-8432-4397-a31a-3f1ba0be66ce None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:01.534766 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ecd4e13f-8432-4397-a31a-3f1ba0be66ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:01.546870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.613651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 336/676] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:01.619786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4e13f-8432-4397-a31a-3f1ba0be66ce None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.636569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1706fbb-2766-4091-a767-81059676a4cf None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.676836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.677515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.677797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.678072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.678808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:01.679116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.681402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.695471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 341/677] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:01.696697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.729916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4e13f-8432-4397-a31a-3f1ba0be66ce None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.762388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.771087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.771822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.772104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.772376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.773111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:01.773539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.774943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.788159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 337/678] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:01.789641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.795785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:01.806143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1ba77-53be-4b8e-a08f-80fce7981ab3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.841237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.842059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.842341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.842615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.850174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.851669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:01.865906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:01.866966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 342/679] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:01.873378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.910639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ca9f-ac9d-4a9f-afce-ebb49f76b472 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.946266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:01.947211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:01.947794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:01.948612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:01.960194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:01.961745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:01.977065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a500-a362-412f-b09a-c412890ca96e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:01.980363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:01.981409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:01.982538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:01.982538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:01.982538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:01.982538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:01.983978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 338/680] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:01.984571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:01.998205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.015687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.016601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.017114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.017389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.027957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.029848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.044000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:02.046141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:02.047154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:02.048227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 343/681] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:02.053811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.094311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9954a2-026f-45a4-a39c-8da504a99b50 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.132265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.133102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.133389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.133661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.141189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.142737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:02.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:02.158409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:02.159284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 339/682] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:02.159802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.173382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dd01b-6bab-45de-9c50-8cf23a8135c7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.209057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.209749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.210035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.210306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.211093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:02.211423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.213175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.226354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 344/683] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:02.227154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.271703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.274233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c5fd4-6f66-4d8e-9e90-c96c5f9371a1 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.292584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.310576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.311249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.311556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.311831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.312555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:02.312863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.314246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.330917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 340/684] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:02.330917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.341675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9318b80-a6ab-43c3-bb72-687e5752bcaa None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.378016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.378924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.379254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.379616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.380538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:02.380920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.382568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.406914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 345/685] 10.222.0.87 () {56 vars in 1135 bytes} [Fri Jun 17 12:39:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:02.407926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.434327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ff82e-66c0-431b-ba3d-3b0a3ca43a1d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.468319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.468988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.469271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.469547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.470336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:02.470647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.472111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.489624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 341/686] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:02.489624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.516078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.520460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7844-81f2-427a-832b-d0dcb97f74b5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.525347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.556793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.557484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.557759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.558035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.558874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=1315c22b7db84afd9ef41bf853b9819b) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:02.559188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.560682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.575412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 346/687] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:02 2022] GET /identity/v3/role_assignments?user.id=1315c22b7db84afd9ef41bf853b9819b&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:02.575412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.604641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795501e6-6299-4885-9209-5648d860fef5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.644846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.645596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.645886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.646163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.647017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.648663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.683994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 342/688] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:39:02 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:02.685068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:02.698305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46a0f1-b7bc-471d-8d27-43cf9a55a0c7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.740993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.742072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.742535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.742983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.813509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.845039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:02.857152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961e2c-fcf8-456a-bc01-1d0595c981d0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:02.866477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.868027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.905328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:02.906675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:02.907156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:02.907637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:02.908688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=40f6ae0e5e164461b4c51b829be9a7d6) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:02.909474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:02.911138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:02.920555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 343/689] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:02 2022] GET /identity/v3/role_assignments?user.id=40f6ae0e5e164461b4c51b829be9a7d6&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:02.997131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 347/690] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:39:03.002748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:03.122435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f597fd9-58f5-435f-96e9-865c92866bfb None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:03.162893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:03.163757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:03.164041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:03.164867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:03.181457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:03.182985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:03.205783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:03.206728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:03.207651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:03.207651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:03.207651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:03.207651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:03.209934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 344/691] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:03.217194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:03.288403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:03.330218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc2335-3382-4ac9-b68e-7b5bcabe8192 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:03.371099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:03.371728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:03.372016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:03.372292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:03.373035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:03.373342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:03.374739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:03.396938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:03.397377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 348/692] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:03.410701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:03.527943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422e7c9-2e1d-4a99-ae55-d1ded4cab34c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:03.579800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:03.580670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:03.580960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:03.581241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:03.590661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:03.592410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:03.613805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:03.614761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:03.615668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:03.618159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 345/693] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:03.624066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:03.757855 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1833a-b716-46bb-a758-f87d52cdb68d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:03.781030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:03.800188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd2438-ac6b-4a71-8697-ad31168909d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:03.800977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd2438-ac6b-4a71-8697-ad31168909d5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:03.801266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd2438-ac6b-4a71-8697-ad31168909d5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:03.801543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd2438-ac6b-4a71-8697-ad31168909d5 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:03.802415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd2438-ac6b-4a71-8697-ad31168909d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:03.802738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd2438-ac6b-4a71-8697-ad31168909d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:03.804251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd2438-ac6b-4a71-8697-ad31168909d5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:03.824480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 349/694] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:03.830698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:03.891929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bb624-34de-412a-b33b-d12bd33abde6 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:03.892233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bb624-34de-412a-b33b-d12bd33abde6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:03.892403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bb624-34de-412a-b33b-d12bd33abde6 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:03.894037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 350/695] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:03.899164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346d28f-f002-4613-bb76-a400b3298076 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:03.899488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346d28f-f002-4613-bb76-a400b3298076 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:03.899773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346d28f-f002-4613-bb76-a400b3298076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:03.917630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3346d28f-f002-4613-bb76-a400b3298076 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:03.919252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3346d28f-f002-4613-bb76-a400b3298076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:03.946420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:03.983150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8adc9-3418-4006-8b5e-a0db884ac1e5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:04.022505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 351/696] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:04.039127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:04.040261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:04.040771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:04.041688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:04.042804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=1a31e162778e4824b7bafd5dcd9b4b4b) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:04.043345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:04.045109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:04.055391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346d28f-f002-4613-bb76-a400b3298076 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:04.060727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 346/697] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:03 2022] GET /identity/v3/role_assignments?user.id=1a31e162778e4824b7bafd5dcd9b4b4b&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:04.181881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346d28f-f002-4613-bb76-a400b3298076 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:04.197217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:04.220301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:04.221183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:04.221466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:04.221752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:04.233466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:04.241237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:04.242142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:04.242977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 352/698] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:04.249017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:04.357319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc38806-4e2b-471e-aa65-58bbdb0be53d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:04.390388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:04.395933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:04.396676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:04.396963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:04.397245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:04.398068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:04.398377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:04.399782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:04.408724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 347/699] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:04.414277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:04.524738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12222-ae16-4f54-b4c2-1f20ee606d73 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:04.561060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:04.561880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:04.562166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:04.562443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:04.574000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:04.575509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:04.593472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:04.595576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:04.597662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:04.597662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:04.597662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:04.597662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:04.597662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 353/700] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:04.600613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:04.704649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efed2f-3285-44ff-8f3e-7809f26c4d0d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:04.747701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:04.748458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:04.748748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:04.749025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:04.749773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:04.750147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:04.751613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:04.764783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 348/701] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:04.770267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:04.799631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:04.879914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeef62-761d-4cf5-8f67-d939c9afccd6 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:04.922020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:04.922851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:04.923135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:04.923428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:04.931895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:04.933435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:04.953992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:04.956194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:04.958470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 354/702] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:39:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:04.961249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:05.035748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:05.079789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693faebd-5071-40bb-a86f-0b2d8a31c281 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:05.118443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd11e9c-4463-450f-9a1f-84e7534535c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:05.123394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11e9c-4463-450f-9a1f-84e7534535c0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.123394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11e9c-4463-450f-9a1f-84e7534535c0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.123394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11e9c-4463-450f-9a1f-84e7534535c0 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.123394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd11e9c-4463-450f-9a1f-84e7534535c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:05.123394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd11e9c-4463-450f-9a1f-84e7534535c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:05.123394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd11e9c-4463-450f-9a1f-84e7534535c0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:05.141656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 349/703] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:39:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:05.147336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:05.224058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd13466-f8cb-450b-868c-8a18a7441b98 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.224409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd13466-f8cb-450b-868c-8a18a7441b98 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.224601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd13466-f8cb-450b-868c-8a18a7441b98 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.226434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 350/704] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:05.233008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c772b-b987-4aa4-861e-03bb04876970 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.233427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c772b-b987-4aa4-861e-03bb04876970 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.233889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c772b-b987-4aa4-861e-03bb04876970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.266074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b94c772b-b987-4aa4-861e-03bb04876970 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:05.267301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b94c772b-b987-4aa4-861e-03bb04876970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:05.296824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:05.315459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9288eb-7f55-458c-8b11-3bac896edb9f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:05.357579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:05.358211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.358503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.358779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.415762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 351/705] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:05.442666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:05.444265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:05.459314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c772b-b987-4aa4-861e-03bb04876970 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:05.544290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:05.596910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:05.618675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c772b-b987-4aa4-861e-03bb04876970 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:05.639073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 355/706] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:39:05.644751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:05.684397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313830f-af05-4a95-94c9-c992911df79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:05.685239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313830f-af05-4a95-94c9-c992911df79b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.685548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313830f-af05-4a95-94c9-c992911df79b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.686065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313830f-af05-4a95-94c9-c992911df79b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.686740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313830f-af05-4a95-94c9-c992911df79b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:05.688910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313830f-af05-4a95-94c9-c992911df79b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-7313830f-af05-4a95-94c9-c992911df79b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:05.708999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:05.710045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:05.710884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:05.715483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 352/707] 10.222.0.87 () {56 vars in 1129 bytes} [Fri Jun 17 12:39:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:05.717405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4bc8b-ea8f-4b2f-8ffe-604172408463 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.717693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4bc8b-ea8f-4b2f-8ffe-604172408463 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.717968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4bc8b-ea8f-4b2f-8ffe-604172408463 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.719414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 353/708] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:05.720607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4bc8b-ea8f-4b2f-8ffe-604172408463 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:05.783763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4dc10-9a76-4ed0-bc8d-9e909b5f7c3d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:05.819612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582a1b0-de4b-42bf-b208-615ef64abf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:05.820578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582a1b0-de4b-42bf-b208-615ef64abf79 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.820862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582a1b0-de4b-42bf-b208-615ef64abf79 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.821131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582a1b0-de4b-42bf-b208-615ef64abf79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.831739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4bc8b-ea8f-4b2f-8ffe-604172408463 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:05.831891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582a1b0-de4b-42bf-b208-615ef64abf79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:05.833211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582a1b0-de4b-42bf-b208-615ef64abf79 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:05.839559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f582a1b0-de4b-42bf-b208-615ef64abf79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:05.850659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:05.851592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:05.852475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:05.852475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:05.852475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:05.852475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:05.852475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 356/709] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:05.853384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de17d-97bf-49fc-b13c-ed8868e929b1 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.853661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de17d-97bf-49fc-b13c-ed8868e929b1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.853956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de17d-97bf-49fc-b13c-ed8868e929b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.869913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-cc7de17d-97bf-49fc-b13c-ed8868e929b1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:05.871089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cc7de17d-97bf-49fc-b13c-ed8868e929b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:05.873471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5272c-0938-4d8f-a08e-e824c81d18b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:05.874506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5272c-0938-4d8f-a08e-e824c81d18b1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.874506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5272c-0938-4d8f-a08e-e824c81d18b1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.874699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5272c-0938-4d8f-a08e-e824c81d18b1 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.875435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5272c-0938-4d8f-a08e-e824c81d18b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:05.875750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5272c-0938-4d8f-a08e-e824c81d18b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:05.877123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5272c-0938-4d8f-a08e-e824c81d18b1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:05.886464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 354/710] 10.222.0.87 () {56 vars in 1124 bytes} [Fri Jun 17 12:39:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:05.886464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da7506-4a67-4f91-a062-7fb0a358dbda None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:05.886464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da7506-4a67-4f91-a062-7fb0a358dbda None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:05.886464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da7506-4a67-4f91-a062-7fb0a358dbda None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:05.886464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 355/711] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:05.886464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da7506-4a67-4f91-a062-7fb0a358dbda None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:05.907743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:05.946041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 357/712] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:05.946952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de17d-97bf-49fc-b13c-ed8868e929b1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.020067 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da7506-4a67-4f91-a062-7fb0a358dbda None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.054872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad453f4-a4c3-41d4-8279-f479e025904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.055527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de17d-97bf-49fc-b13c-ed8868e929b1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.055641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad453f4-a4c3-41d4-8279-f479e025904e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.055864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad453f4-a4c3-41d4-8279-f479e025904e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.056138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad453f4-a4c3-41d4-8279-f479e025904e None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.056869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad453f4-a4c3-41d4-8279-f479e025904e None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:06.057443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad453f4-a4c3-41d4-8279-f479e025904e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.058819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad453f4-a4c3-41d4-8279-f479e025904e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.070728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 356/713] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:06.072164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c88b-5009-41db-a5bd-0d054a36b13b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.072590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c88b-5009-41db-a5bd-0d054a36b13b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.073051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c88b-5009-41db-a5bd-0d054a36b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.090058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7042c88b-5009-41db-a5bd-0d054a36b13b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:06.091249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7042c88b-5009-41db-a5bd-0d054a36b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:06.098022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.099115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.099423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.099707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.100424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:06.101097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.102824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.108800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 358/714] 10.222.0.87 () {56 vars in 1124 bytes} [Fri Jun 17 12:39:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:06.112496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.128854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:06.152635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:06.199128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 357/715] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:06.201383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c88b-5009-41db-a5bd-0d054a36b13b None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.240858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338628c-876e-4acd-ac49-fde0b7d16e10 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.288396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.289917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.290834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.292216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] PATH_INFO: `/v3/services/network` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.293083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.294903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:06.316846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:06.318192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:06.319052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:06.321365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 359/716] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:06.321541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.344504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c88b-5009-41db-a5bd-0d054a36b13b None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.389850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.390616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.390903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.391178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.400716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.402585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:06.422447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:06.423395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:06.424329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 358/717] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:06.425370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.455737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:06.462413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:06.467354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db02ab-841f-4408-b2cf-c0668b842cd5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.521875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.523067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.523614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.524090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.539226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:06.550960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:06.551951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:06.552791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:06.553457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b875e-a581-46a8-b4a6-dad367014378 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.556177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 360/718] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:06.556835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.593276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.594988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.595542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.596235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.597707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] RBAC: Adding query filter params (name=network) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:06.598293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.600476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.611957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 359/719] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:06.612561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.694759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15244-4178-434b-8721-91c0a8500d15 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.711766 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:06.718796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261823e-8dc7-480a-a6e5-ae007e6a74ed None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.738012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.738909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.739347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.740239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.741474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:06.741937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.743976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.757067 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.758070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.758555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.759033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.760031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:06.760648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.761653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 361/720] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:06.762528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.762636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.773438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 360/721] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:06.774964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.779284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:06.873877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ea44f-cc5c-46f5-b697-6121271acfc1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.899415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7df66-f068-498e-b428-97495b92dcf3 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:06.921009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.922465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.923053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.923641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.924758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] RBAC: Adding query filter params (type=network) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:06.925216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.927436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.943422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 362/722] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:06.943422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:06.948922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceaecd-b2be-4ad6-81c1-b9985ff578a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:06.949476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceaecd-b2be-4ad6-81c1-b9985ff578a0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:06.950474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceaecd-b2be-4ad6-81c1-b9985ff578a0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:06.950770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceaecd-b2be-4ad6-81c1-b9985ff578a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:06.953075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceaecd-b2be-4ad6-81c1-b9985ff578a0 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=abaa59b4e88642c681e9563295f6fad9) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:06.953565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceaecd-b2be-4ad6-81c1-b9985ff578a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:06.955335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceaecd-b2be-4ad6-81c1-b9985ff578a0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:06.970592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 361/723] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/role_assignments?user.id=abaa59b4e88642c681e9563295f6fad9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:07.017093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad76f-4263-4516-a2c6-c0108582b816 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.017449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad76f-4263-4516-a2c6-c0108582b816 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.017742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad76f-4263-4516-a2c6-c0108582b816 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.019395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 362/724] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:39:07.025518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cc1ce-3366-4f8f-9328-34d3cf02f8b4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.026304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cc1ce-3366-4f8f-9328-34d3cf02f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.026824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cc1ce-3366-4f8f-9328-34d3cf02f8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.055454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f77cc1ce-3366-4f8f-9328-34d3cf02f8b4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:07.058444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f77cc1ce-3366-4f8f-9328-34d3cf02f8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:07.065264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:07.118637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:07.176210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 363/725] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 12:39:07.196192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24947-8038-4d6a-9ef2-fcf15cdd79d3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:07.206860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cc1ce-3366-4f8f-9328-34d3cf02f8b4 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:07.244069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:07.245711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.246372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.246849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.260973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:07.263350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:07.287773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:07.288921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:07.290038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:07.290038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:07.290038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:07.290038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:07.293010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 363/726] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:07.298806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:07.367423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cc1ce-3366-4f8f-9328-34d3cf02f8b4 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:07.447052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf52a3-1525-4056-94a5-c3e7367e392a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:07.479853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:07.488101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:07.489058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.489550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.490069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.491003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:07.491545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:07.493259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:07.507041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 364/727] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:07.508039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:07.509112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.509591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.510074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.514190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:07.532573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:07.537409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:07.546305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:07.547238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:07.548136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:07.550905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 364/728] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:07.556931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:07.686811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ae5a-416d-4709-82d1-826e459e959c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:07.721262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f7cef-fbf5-4de5-b146-774979f0855b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:07.748085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:07.748770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.749049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.749321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.750076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:07.750381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:07.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:07.765907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 365/729] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:07.776381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:07.783062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:07.783934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.784220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.784500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.793562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:07.795068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:07.811471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:07.812489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:07.813413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 365/730] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:39:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:07.818518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:07.875382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:07.928872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f1604-5743-4d9e-8412-d393a2973d63 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:07.957220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:07.964883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:07.965581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:07.965871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:07.966150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:07.967008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:07.967318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:07.968952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:07.991754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 366/731] 10.222.0.87 () {56 vars in 1149 bytes} [Fri Jun 17 12:39:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:07.993316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:08.062201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc645e-0fd6-48d4-99ba-b4ce290c450e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:08.095002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df50588-c9e3-4c68-94b9-11ce54d494de None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:08.104677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:08.130686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:08.131358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.131667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.131946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:08.196909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.197188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.197461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.201883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:08.203399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:08.240325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:08.241815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:08.283896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:08.284841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:08.285706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:08.285706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:08.285706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:08.285706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:08.285706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 366/732] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 509 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:08.296377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:08.337682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 367/733] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 12:39:08.343233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:08.359762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:08.477736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97939f-8c4a-4f30-b688-524832689a13 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:08.486852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02793bad-0253-44bd-a700-6d155f5ae575 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:08.496146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:08.523119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83241-75ef-4023-a548-446b0173c654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:08.524146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83241-75ef-4023-a548-446b0173c654 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.524508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83241-75ef-4023-a548-446b0173c654 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.524926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83241-75ef-4023-a548-446b0173c654 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.525293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef8c83-5da2-4799-b754-fd43358cbb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:08.525814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83241-75ef-4023-a548-446b0173c654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:08.526262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef8c83-5da2-4799-b754-fd43358cbb14 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.526476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83241-75ef-4023-a548-446b0173c654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:08.526719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef8c83-5da2-4799-b754-fd43358cbb14 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.527057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef8c83-5da2-4799-b754-fd43358cbb14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.528196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83241-75ef-4023-a548-446b0173c654 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:08.538542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef8c83-5da2-4799-b754-fd43358cbb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:08.540204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef8c83-5da2-4799-b754-fd43358cbb14 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:08.544264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 367/734] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:08.545650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b39fc-b1f1-4a13-9c12-99e81529fa43 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.546027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b39fc-b1f1-4a13-9c12-99e81529fa43 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.546370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b39fc-b1f1-4a13-9c12-99e81529fa43 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.548452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 368/735] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:08.550018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b39fc-b1f1-4a13-9c12-99e81529fa43 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-98ef8c83-5da2-4799-b754-fd43358cbb14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:08.563752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:08.564997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:08.567138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:08.567138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:08.567138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:08.567138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:08.571412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 368/736] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:08.571412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61d2d1-933c-4782-abc4-2f6f32c7e7c3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.571412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61d2d1-933c-4782-abc4-2f6f32c7e7c3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.571412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61d2d1-933c-4782-abc4-2f6f32c7e7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.588600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8a61d2d1-933c-4782-abc4-2f6f32c7e7c3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:08.589828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8a61d2d1-933c-4782-abc4-2f6f32c7e7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:08.654594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:08.667594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 369/737] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:08.668585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61d2d1-933c-4782-abc4-2f6f32c7e7c3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:08.683780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b39fc-b1f1-4a13-9c12-99e81529fa43 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:08.735673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:08.736771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.737206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.737624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] PATH_INFO: `/v3/users/swift` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.745249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:08.746890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:08.762172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:08.763341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:08.764461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:08.766636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 369/738] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:08.767682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:08.783511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61d2d1-933c-4782-abc4-2f6f32c7e7c3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:08.805805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:08.831131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:08.831869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.832176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.832685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.833488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:08.833799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:08.835771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:08.859531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 370/739] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:08.860367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:08.882608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345c9b7-e219-495d-85e6-bf9b1580406a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:08.919189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ab52-9050-4257-8661-498b51901faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:08.920314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ab52-9050-4257-8661-498b51901faf None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:08.920823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ab52-9050-4257-8661-498b51901faf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:08.921296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ab52-9050-4257-8661-498b51901faf None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:08.922246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634ab52-9050-4257-8661-498b51901faf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:08.922934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:08.924405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634ab52-9050-4257-8661-498b51901faf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:08.948191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 370/740] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:08 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:08.949434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ab52-9050-4257-8661-498b51901faf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.024495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3dd92-86ef-4436-ae7d-73f3174d3714 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:09.070440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:09.097094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ab52-9050-4257-8661-498b51901faf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:09.135393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:09.136211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.136581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.136808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.137612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:09.137926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:09.139419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:09.142018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:09.143081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.143609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.144087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.153404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:09.155285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:09.173298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:09.174277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:09.175159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:09.178208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 371/741] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:09.187543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.193887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 371/742] 10.222.0.87 () {56 vars in 1135 bytes} [Fri Jun 17 12:39:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:09.208491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.321059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280df44-f038-4c59-8b5d-882b6f3b7f7a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:09.353343 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222b66f-b3bf-4f6c-9f89-bc4000f02e8c None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:09.362979 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce84a47-c1da-4224-9575-ff7e187b7a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:09.363940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce84a47-c1da-4224-9575-ff7e187b7a6b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.364371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce84a47-c1da-4224-9575-ff7e187b7a6b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.364665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:09.364976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce84a47-c1da-4224-9575-ff7e187b7a6b None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.365931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce84a47-c1da-4224-9575-ff7e187b7a6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:09.366388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce84a47-c1da-4224-9575-ff7e187b7a6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:09.368006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce84a47-c1da-4224-9575-ff7e187b7a6b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:09.384854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 372/743] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:09.386916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef3e8-bb05-4aca-b442-ab3725de2c8d None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.391478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef3e8-bb05-4aca-b442-ab3725de2c8d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.391478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef3e8-bb05-4aca-b442-ab3725de2c8d None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.391478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 373/744] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:09.395233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bef3e8-bb05-4aca-b442-ab3725de2c8d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.406277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6bc1b-991e-4e5d-8316-817fece2657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:09.406976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6bc1b-991e-4e5d-8316-817fece2657c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.407257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6bc1b-991e-4e5d-8316-817fece2657c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.407687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6bc1b-991e-4e5d-8316-817fece2657c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.410190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:09.485619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6bc1b-991e-4e5d-8316-817fece2657c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:09.487087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6bc1b-991e-4e5d-8316-817fece2657c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:09.519145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bef3e8-bb05-4aca-b442-ab3725de2c8d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:09.553898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ccb70-6823-47db-8c6d-b21189348e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:09.554570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ccb70-6823-47db-8c6d-b21189348e98 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.554858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ccb70-6823-47db-8c6d-b21189348e98 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.555138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ccb70-6823-47db-8c6d-b21189348e98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.556333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ccb70-6823-47db-8c6d-b21189348e98 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=cac0c2cb80a34d608aba65ba0c628f24) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:09.556333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ccb70-6823-47db-8c6d-b21189348e98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:09.557838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ccb70-6823-47db-8c6d-b21189348e98 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:09.571403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 374/745] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:09 2022] GET /identity/v3/role_assignments?user.id=cac0c2cb80a34d608aba65ba0c628f24&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:09.571403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2777c77-c517-4687-82c1-0cdd812887f1 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.571403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2777c77-c517-4687-82c1-0cdd812887f1 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.571403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2777c77-c517-4687-82c1-0cdd812887f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.588484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f2777c77-c517-4687-82c1-0cdd812887f1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:09.590562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f2777c77-c517-4687-82c1-0cdd812887f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:09.608227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 372/746] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:09 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 12:39:09.610384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ff1d7-077e-4431-8743-9aca2cbcd794 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.610781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ff1d7-077e-4431-8743-9aca2cbcd794 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.611158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ff1d7-077e-4431-8743-9aca2cbcd794 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.613511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 373/747] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:09.614435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ff1d7-077e-4431-8743-9aca2cbcd794 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.631808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:09.673522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 375/748] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:09.675037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d350e-9788-4104-b625-a59948280aef None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.675338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d350e-9788-4104-b625-a59948280aef None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.675666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d350e-9788-4104-b625-a59948280aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.698492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:09.702661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-894d350e-9788-4104-b625-a59948280aef None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:09.704009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-894d350e-9788-4104-b625-a59948280aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:09.760182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ff1d7-077e-4431-8743-9aca2cbcd794 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:09.785671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 376/749] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 12:39:09.786690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d350e-9788-4104-b625-a59948280aef None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.823117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:09.824052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.824341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.824618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.836368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:09.838130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:09.856429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:09.857346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:09.858202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:09.858202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:09.858202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:09.858202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:09.858202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 374/750] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:09.861363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.893449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d350e-9788-4104-b625-a59948280aef None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:09.928301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:09.930757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:09.931716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:09.932033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:09.932490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:09.934563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:09.945014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:09.952958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:09.953909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:09.954723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 377/751] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:09.955428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:09.986852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005195a-5bb3-43f8-a1ff-e57c1430f351 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.033145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.033854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.034138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.034414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.035153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.036630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.058990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 375/752] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:09 2022] POST /identity/v3/services => generated 240 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:10.063753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:10.087077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d490c-b13d-4c3a-a0f1-75c48a7cc92b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.184036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.184693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.184984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.185263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.187103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:10.187441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.189224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.213219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 378/753] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:10.219591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:10.273933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae4f63-ec8b-415e-be2c-8a64b3c9eb67 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.297523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:10.307845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:10.317723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.318684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.318972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.319250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.320064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:10.320384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.323282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.337970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 376/754] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:10.341149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:10.373779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9f054-3294-47e7-9361-d88a1caf2623 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.417118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.417938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.418221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.418495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.427103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.429423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:10.446735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:10.447695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:10.448891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 379/755] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:10.453734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:10.468788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25da7a-3791-4bcd-b3a0-5c5b2d4ee04e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.525036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.526791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.527548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.528798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.551124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.553489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:10.586435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:10.587419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:10.588425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:10.588425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:10.588425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:10.588425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:10.592640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 377/756] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:10.598740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:10.633490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ac1d-a55a-4fe9-a7df-a69008352946 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.650692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:10.659112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:10.696548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.697689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.699453 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.699890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.700840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:10.701488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.703873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.732977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 380/757] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:10.734100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:10.784448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803357-486d-4399-a110-00c3c1f300dd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.872150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.873137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.873570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.874004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.874118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636983ec-9d66-4f77-afcc-2b7639a72bdd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:10.875003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:10.875486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.877041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.902063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 378/758] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:10.908034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:10.910189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:10.910931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:10.911280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:10.915397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:10.915397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=1a31e162778e4824b7bafd5dcd9b4b4b) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:10.915397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:10.915397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:10.921114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 381/759] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:10 2022] GET /identity/v3/role_assignments?user.id=1a31e162778e4824b7bafd5dcd9b4b4b&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:11.072466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:11.079675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555419ea-8dd7-43a7-9863-614fdd1fe882 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:11.136224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:11.137157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:11.137447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:11.137719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:11.145948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:11.147450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:11.165195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:11.166212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:11.167781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 379/760] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:39:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:11.175262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:11.220547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:11.297608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24a132-4852-41bb-b36a-5d0c1999f043 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:11.345631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5857577-a218-4322-a715-7f50e8dbbd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:11.346249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857577-a218-4322-a715-7f50e8dbbd98 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:11.346536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857577-a218-4322-a715-7f50e8dbbd98 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:11.346817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857577-a218-4322-a715-7f50e8dbbd98 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:11.347720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5857577-a218-4322-a715-7f50e8dbbd98 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:11.348038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5857577-a218-4322-a715-7f50e8dbbd98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:11.349680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5857577-a218-4322-a715-7f50e8dbbd98 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:11.383179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 382/761] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:39:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:11.389383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:11.540960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568acde6-99f4-4f90-bca4-e58442d981ff None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:11.576840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a4f4-4d3f-4525-bd09-9a581cb3a6c5 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:11.577112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a4f4-4d3f-4525-bd09-9a581cb3a6c5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:11.577400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a4f4-4d3f-4525-bd09-9a581cb3a6c5 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:11.580297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 383/762] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:11.585723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:11.586553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbea8ec-b319-491d-822b-9d728f2176fc None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:11.586678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:11.586842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbea8ec-b319-491d-822b-9d728f2176fc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:11.586935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:11.587128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbea8ec-b319-491d-822b-9d728f2176fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:11.587228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:11.599361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:11.600794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:11.619009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ffbea8ec-b319-491d-822b-9d728f2176fc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:11.620314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ffbea8ec-b319-491d-822b-9d728f2176fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:11.637243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:11.638197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:11.639075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:11.639075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:11.639075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:11.639075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:11.639075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 380/763] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:11.645491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:11.657466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:11.740769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 384/764] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:11.774774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbea8ec-b319-491d-822b-9d728f2176fc None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:11.815841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217a3d8-5cc2-4d7c-9401-308435bd5f97 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:11.819725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:11.853734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:11.854840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:11.855328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:11.855828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:11.856775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:11.857290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:11.858890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:11.871612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 381/765] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:11.876612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:11.914822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbea8ec-b319-491d-822b-9d728f2176fc None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:11.953827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:11.954676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:11.954958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:11.955235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:11.955948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:11.957752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:11.971984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:11.972984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:11.974287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 385/766] 10.222.0.87 () {56 vars in 1129 bytes} [Fri Jun 17 12:39:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:11.980546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.058327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:12.084906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:12.105989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd331-c28d-452f-bd98-c3a480a24a9e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.107802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3d15a-54c8-4b11-b2bc-590e02d50326 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.142039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2523-4b20-4489-842f-a38a1dc89ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.143031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2523-4b20-4489-842f-a38a1dc89ba1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.143598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2523-4b20-4489-842f-a38a1dc89ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.144078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2523-4b20-4489-842f-a38a1dc89ba1 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.144990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2523-4b20-4489-842f-a38a1dc89ba1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:12.145534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2523-4b20-4489-842f-a38a1dc89ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:12.147225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2523-4b20-4489-842f-a38a1dc89ba1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:12.150867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f270664-f2fa-4677-868b-94fcec54ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.152012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f270664-f2fa-4677-868b-94fcec54ba1b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.152299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f270664-f2fa-4677-868b-94fcec54ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.152577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f270664-f2fa-4677-868b-94fcec54ba1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.153716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 386/767] 10.222.0.87 () {56 vars in 1124 bytes} [Fri Jun 17 12:39:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:12.155196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985001-de14-46d7-99fd-de92ed4b540d None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.155717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985001-de14-46d7-99fd-de92ed4b540d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.156217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985001-de14-46d7-99fd-de92ed4b540d None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.162131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 387/768] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:12.163129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985001-de14-46d7-99fd-de92ed4b540d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.164397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f270664-f2fa-4677-868b-94fcec54ba1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:12.165858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f270664-f2fa-4677-868b-94fcec54ba1b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-0f270664-f2fa-4677-868b-94fcec54ba1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:12.185494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:12.186417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:12.187291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:12.191803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 382/769] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:12.193217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf11ad-3b1e-4e72-af98-b9dab313301d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.193507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf11ad-3b1e-4e72-af98-b9dab313301d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.193790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf11ad-3b1e-4e72-af98-b9dab313301d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.216436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-48bf11ad-3b1e-4e72-af98-b9dab313301d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:12.217667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-48bf11ad-3b1e-4e72-af98-b9dab313301d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:12.286948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985001-de14-46d7-99fd-de92ed4b540d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.308282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 383/770] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:12.309312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf11ad-3b1e-4e72-af98-b9dab313301d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.323533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.324400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.324770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.325051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.325774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:12.326080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:12.327593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:12.334276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 388/771] 10.222.0.87 () {56 vars in 1124 bytes} [Fri Jun 17 12:39:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:12.341036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.363216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:12.376311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:12.428222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf11ad-3b1e-4e72-af98-b9dab313301d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.464102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.464873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.465220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.465555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.466420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:12.466793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:12.468355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:12.475050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cb046-c32c-42a4-bc69-bb49eaaed9f0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.487661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 384/772] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:12.488717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.517723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.518649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.518937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.519291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.520764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] RBAC: Adding query filter params (interface=public, service_id=3dbc2daa839749719a49d788a17139c8, region_id=RegionOne) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:12.521083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:12.523077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:12.531443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 389/773] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:39:12 2022] GET /identity/v3/endpoints?service_id=3dbc2daa839749719a49d788a17139c8&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:12.532355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.625803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8052c0f-1f7f-4de1-9c76-bfb90aa8f5bb None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.631904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:12.645712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:12.662047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7d70d-3567-4ba1-9cba-d781e54fe4e2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.663853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.664740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.665686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.665686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.676286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:12.684228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:12.685158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:12.685973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:12.690727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 385/774] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:39:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:12.691759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.704284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.705562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.706014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.706457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/abaa59b4e88642c681e9563295f6fad9/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.708849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=abaa59b4e88642c681e9563295f6fad9, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:12.710539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:12.728767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 390/775] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:39:12 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/abaa59b4e88642c681e9563295f6fad9/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:39:12.730348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:12.833713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1b727-32c1-4d60-80a9-01e33c6d4762 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:12.894255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c867f6a-d8d1-4e02-a4c3-fa2bced76544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:12.895772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867f6a-d8d1-4e02-a4c3-fa2bced76544 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.896255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867f6a-d8d1-4e02-a4c3-fa2bced76544 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.896966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867f6a-d8d1-4e02-a4c3-fa2bced76544 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.897843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c867f6a-d8d1-4e02-a4c3-fa2bced76544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:12.899737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c867f6a-d8d1-4e02-a4c3-fa2bced76544 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:12.911670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 386/776] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:12 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:12.914596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b231e-0470-4ec8-8e0c-bc388a5e9180 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.914926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b231e-0470-4ec8-8e0c-bc388a5e9180 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.915189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b231e-0470-4ec8-8e0c-bc388a5e9180 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.921589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 387/777] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:39:12.923505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e526b-f424-46c4-8fc2-98e687bdf3a0 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:12.923805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e526b-f424-46c4-8fc2-98e687bdf3a0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:12.924091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e526b-f424-46c4-8fc2-98e687bdf3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:12.942794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9a1e526b-f424-46c4-8fc2-98e687bdf3a0 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:12.944079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9a1e526b-f424-46c4-8fc2-98e687bdf3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:12.993420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203920b5-0962-4004-bf7d-bbff01f0cc46 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.016083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:13.050462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.051173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.051502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.051784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.052552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:13.052866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.054254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.061989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:13.067049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 391/778] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:39:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:13.071688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 388/779] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:13.072452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.092197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e526b-f424-46c4-8fc2-98e687bdf3a0 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.237109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db136ec-08f0-4dd4-80fe-a08dfbc087d6 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.262380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e526b-f424-46c4-8fc2-98e687bdf3a0 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.305870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd4aaf-44fe-46a4-bb8f-470e944929da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.307252 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd4aaf-44fe-46a4-bb8f-470e944929da None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.308188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd4aaf-44fe-46a4-bb8f-470e944929da None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.308756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd4aaf-44fe-46a4-bb8f-470e944929da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.310499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.315413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.315413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.315413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.315413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.315413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:13.328887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:13.330129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:13.331125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:13.335838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 389/780] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:13.337404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.342397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd4aaf-44fe-46a4-bb8f-470e944929da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.344107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd4aaf-44fe-46a4-bb8f-470e944929da None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-30cd4aaf-44fe-46a4-bb8f-470e944929da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:13.373349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:13.374495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:13.375607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:13.375607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:13.375607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:13.375607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:13.381141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 392/781] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:13.382611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216169f-4631-4f22-ad29-cfe395013263 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.383146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216169f-4631-4f22-ad29-cfe395013263 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.383535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216169f-4631-4f22-ad29-cfe395013263 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.385236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 393/782] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:13.390188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216169f-4631-4f22-ad29-cfe395013263 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.394153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:13.468654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:13.479148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e328f-61b7-43d1-b58e-c9d9c52305f6 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.514992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58b0b-2c6d-4715-ab66-5cb16c928376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.515740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58b0b-2c6d-4715-ab66-5cb16c928376 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.516025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58b0b-2c6d-4715-ab66-5cb16c928376 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.516302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58b0b-2c6d-4715-ab66-5cb16c928376 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.517019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d58b0b-2c6d-4715-ab66-5cb16c928376 None admin] RBAC: Adding query filter params (name=network) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:13.517329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d58b0b-2c6d-4715-ab66-5cb16c928376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.518716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d58b0b-2c6d-4715-ab66-5cb16c928376 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.524081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 390/783] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:13.525352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9836-d438-4347-85aa-74c944f6a60c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.525642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9836-d438-4347-85aa-74c944f6a60c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.525923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9836-d438-4347-85aa-74c944f6a60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.537007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216169f-4631-4f22-ad29-cfe395013263 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.543762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-83fb9836-d438-4347-85aa-74c944f6a60c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:13.545170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-83fb9836-d438-4347-85aa-74c944f6a60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:13.582857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.583614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.583901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.584182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.584950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:13.585263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.586703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.603664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 394/784] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:13.604745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.624399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 391/785] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:13.625413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9836-d438-4347-85aa-74c944f6a60c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.654956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:13.726209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df8153-b02f-4e1f-8a1c-cccbff94b212 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.731596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9836-d438-4347-85aa-74c944f6a60c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.739886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:13.763107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.763909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.764196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.764476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.765249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] RBAC: Adding query filter params (type=network) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:13.765502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.766946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.768243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.769144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.769492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.769832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.777096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 395/786] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:13.777740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.778050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.779810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:13.796081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:13.798257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:13.800132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:13.807393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 392/787] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:13.807393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.886983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd0f8-c906-469a-9b27-6b273e9acc92 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.896957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:13.910112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157790b0-bbb5-4768-b0bf-3512e8183595 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:13.926934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.927781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.928075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.928355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.939958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:13.947206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:13.947932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:13.948109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:13.950574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:13.952538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:13.954050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:13.954050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:13.954050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] RBAC: Adding query filter params (interface=public, service_id=7740ee89ba1a42d28646220c46793c46, region_id=RegionOne) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:13.954286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 396/788] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:13.954366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:13.954445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:13.954538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:13.967472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 393/789] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/endpoints?service_id=7740ee89ba1a42d28646220c46793c46&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:13.968466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.006118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:14.095340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233b70-8f1a-448d-8d36-46df873fc655 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.107748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c147a5-e3ae-4fa9-bb6c-4e19c0027715 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.147310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.148018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.148294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.148565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.149577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:14.150190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.152783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:14.159132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.160108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.160592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.161312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.162504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:14.163027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.165389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:14.180232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 397/791] 10.222.0.87 () {56 vars in 1149 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:14.180232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.180619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 394/791] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:14.181650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.201243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:14.300038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:14.310787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58dbc-2b30-47e6-9a73-2428a3bbe4ae None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.314167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0d929-9c9b-4fb6-babf-2b9931ba74e1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.348740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.349478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.349772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.350074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.353791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.354761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.355422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.355898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.356768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.358846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:14.361628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.363079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:14.371661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 398/792] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:14 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:14.374269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:14.399979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:14.400930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:14.402910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:14.402910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:14.402910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:14.402910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:14.402910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 395/793] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:14.407134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.535964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:14.554111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69438180-a895-42d4-bd8e-7ac130998978 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.613737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdb476-6b94-41e5-9de5-9239f0feb442 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.617438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.618521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.619019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.619532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.634281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.636759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:14.653951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab2def-0335-41de-946a-17f518048ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.654628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2def-0335-41de-946a-17f518048ace None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.654910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2def-0335-41de-946a-17f518048ace None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.655182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2def-0335-41de-946a-17f518048ace None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.655943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab2def-0335-41de-946a-17f518048ace None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:14.658039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab2def-0335-41de-946a-17f518048ace None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.658039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab2def-0335-41de-946a-17f518048ace None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:14.663554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:14.664414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:14.665280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:14.665280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:14.665280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:14.665280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:14.671307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 396/795] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:14.671541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 399/795] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:14.674328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab2def-0335-41de-946a-17f518048ace None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.676027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.692851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:14.828415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328395d9-b9c0-490a-871f-01dc0e904f09 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.868386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.869554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.870266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.871867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.884861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.887574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:14.894217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab2def-0335-41de-946a-17f518048ace None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:14.916214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:14.917965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:14.919038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 400/796] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:14.924828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:14.967322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:14.983785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:14.984309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:14.984599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:14.984982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:14.985828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:14.986128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:14.987563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:15.016976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 397/797] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:15.022651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:15.058012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:15.117504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbde0e-f066-4059-9ecb-50444b364890 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:15.191840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316133bf-05f8-4cef-a4ef-8cd42c81da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:15.192529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316133bf-05f8-4cef-a4ef-8cd42c81da95 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.192820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316133bf-05f8-4cef-a4ef-8cd42c81da95 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.193096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316133bf-05f8-4cef-a4ef-8cd42c81da95 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.193910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316133bf-05f8-4cef-a4ef-8cd42c81da95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:15.194216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316133bf-05f8-4cef-a4ef-8cd42c81da95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:15.195683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316133bf-05f8-4cef-a4ef-8cd42c81da95 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:15.223692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afd655-e443-4db7-84f0-276e88296210 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:15.228609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 401/798] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:15.230213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a505f1-2896-4f87-94e5-cefc06e174cb None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.230502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a505f1-2896-4f87-94e5-cefc06e174cb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.230777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a505f1-2896-4f87-94e5-cefc06e174cb None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.232307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 402/799] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:15.232988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a505f1-2896-4f87-94e5-cefc06e174cb None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:15.265315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcd062-0b91-47be-81f3-8c0ef2a3f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:15.266138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcd062-0b91-47be-81f3-8c0ef2a3f60d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.266425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcd062-0b91-47be-81f3-8c0ef2a3f60d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.266703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcd062-0b91-47be-81f3-8c0ef2a3f60d None admin] PATH_INFO: `/v3/users/swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.276296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcd062-0b91-47be-81f3-8c0ef2a3f60d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:15.278301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcd062-0b91-47be-81f3-8c0ef2a3f60d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-4bfcd062-0b91-47be-81f3-8c0ef2a3f60d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:15.294762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:15.295816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:15.296717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 398/800] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:15.302672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c3b29-d435-438a-b4b8-a50bc132f677 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.302672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c3b29-d435-438a-b4b8-a50bc132f677 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.302672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c3b29-d435-438a-b4b8-a50bc132f677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.329395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8d8c3b29-d435-438a-b4b8-a50bc132f677 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:15.332244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8d8c3b29-d435-438a-b4b8-a50bc132f677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:15.357336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a505f1-2896-4f87-94e5-cefc06e174cb None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:15.389036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:15.393833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:15.397220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f8571-13bf-41ba-abe1-a3f2944feb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:15.398188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f8571-13bf-41ba-abe1-a3f2944feb2d None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.398470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f8571-13bf-41ba-abe1-a3f2944feb2d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.398744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f8571-13bf-41ba-abe1-a3f2944feb2d None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/cac0c2cb80a34d608aba65ba0c628f24/roles/e381464c740146119ff513d42bb0044b` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.414159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f8571-13bf-41ba-abe1-a3f2944feb2d None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=cac0c2cb80a34d608aba65ba0c628f24, role_id=e381464c740146119ff513d42bb0044b)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:15.415704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f8571-13bf-41ba-abe1-a3f2944feb2d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:15.422215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 399/801] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:15.423045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c3b29-d435-438a-b4b8-a50bc132f677 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:15.441610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 403/802] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:39:15 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/cac0c2cb80a34d608aba65ba0c628f24/roles/e381464c740146119ff513d42bb0044b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:39:15.443492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552eef4-a96e-49ea-8228-0e69b595d081 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.443786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552eef4-a96e-49ea-8228-0e69b595d081 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.444066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552eef4-a96e-49ea-8228-0e69b595d081 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.446693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 404/803] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:15.452425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001adca-25c5-4faa-b48f-af4fdd7b7694 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.453039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001adca-25c5-4faa-b48f-af4fdd7b7694 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.454922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001adca-25c5-4faa-b48f-af4fdd7b7694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.473966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3001adca-25c5-4faa-b48f-af4fdd7b7694 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:15.475387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3001adca-25c5-4faa-b48f-af4fdd7b7694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:15.557695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c3b29-d435-438a-b4b8-a50bc132f677 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:15.558866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 405/804] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:15.560232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001adca-25c5-4faa-b48f-af4fdd7b7694 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:15.598791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:15.599568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.599853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.600126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.601012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:15.601326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:15.602873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:15.628391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 400/805] 10.222.0.87 () {56 vars in 1135 bytes} [Fri Jun 17 12:39:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:15.628391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:15.663190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:15.685910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001adca-25c5-4faa-b48f-af4fdd7b7694 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:15.702242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:15.728965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:15.730025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.730539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.730919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.743941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:15.751774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:15.752951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:15.754028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 406/806] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:15.754712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:15.759316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2d985-e9e6-43e4-86cb-22c0f16bd176 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:15.880520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357f615-3e3f-410d-93b3-8a7d1af85409 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:15.920256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:15.921000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.921281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.921556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.922218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:15.923644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:15.924417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:15.925643 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:15.926071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:15.926815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:15.937316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:15.938341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:15.939337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 401/807] 10.222.0.87 () {56 vars in 1129 bytes} [Fri Jun 17 12:39:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:15.940456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:15.943472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:15.946152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:15.969448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:15.970386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:15.971284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:15.971284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:15.971284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:15.971284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:15.977348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 407/808] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:15.977348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.002181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.062236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.066757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ee1a-5233-400f-9bfe-7c3b4ea91e15 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.080015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843940a-47ad-475e-be91-2145da1d62f2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.103811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.104846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.105331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.105850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.106920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:16.107484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.109243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.114281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.114973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.115156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.115493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.116308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:16.116465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.117833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.122924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 408/809] 10.222.0.87 () {56 vars in 1124 bytes} [Fri Jun 17 12:39:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:16.123755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.131496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 402/810] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:16.132669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.216090 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.231007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194f4ed-a91e-4d7a-91e8-1b8c0223eaf2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.241110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e66acd-4095-40b6-85a3-1d9c096700e8 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.268078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.268757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.269047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.269327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.270222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:16.270399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.271795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.278827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.279950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.280431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.281057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.281968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:16.282538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.284164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.288865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 409/811] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:16.288865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.295737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 403/812] 10.222.0.87 () {56 vars in 1124 bytes} [Fri Jun 17 12:39:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:16.297087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.327860 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.397114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc59d4-acda-48b9-a7c3-13c4f84c35c9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.425049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f8cb-d1d8-44f1-96e1-50c256a7c2a3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.433937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.435008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.435253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.435657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.446497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.448112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.455694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:16.468486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:16.469548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:16.470645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:16.470645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:16.470645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:16.470645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:16.471582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.472551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.473041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.473443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.476966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 410/813] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:16.477938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.481730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.484371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:16.501754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:16.502825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:16.503957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:16.511206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 404/814] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:16.511206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.593501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a5a31-4d9e-44c0-8342-6ba26c3e39a9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.622376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.632527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.633206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.633500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.633956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.634524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.636054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.644285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5540-3f86-41d9-b43c-e0b2b5ffd52e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.663566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 411/815] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:16.664596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.690676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.691812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.692422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.692910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.693894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:16.694415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.696215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.711181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 405/816] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:16.723016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.770339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.829079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f698c-6b3b-4ee5-8aaf-d93dca9000ac None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.866968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0cd0f-d359-4067-a5a3-2c75a7ae1224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.867978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cd0f-d359-4067-a5a3-2c75a7ae1224 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.868404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cd0f-d359-4067-a5a3-2c75a7ae1224 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.868820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cd0f-d359-4067-a5a3-2c75a7ae1224 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.869761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0cd0f-d359-4067-a5a3-2c75a7ae1224 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:16.870622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0cd0f-d359-4067-a5a3-2c75a7ae1224 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.872313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0cd0f-d359-4067-a5a3-2c75a7ae1224 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.873172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ffe52-9c7a-44f0-8060-08846b2448c9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:16.890500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 412/817] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:16.890500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e765a5-5901-401e-a127-50105e2a28b5 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.890500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e765a5-5901-401e-a127-50105e2a28b5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.890500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e765a5-5901-401e-a127-50105e2a28b5 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.891678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 413/818] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:16.892365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e765a5-5901-401e-a127-50105e2a28b5 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:16.914509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a99bd9-5e9b-4dbc-a9b5-e6db799076dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:16.915319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a99bd9-5e9b-4dbc-a9b5-e6db799076dd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.915620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a99bd9-5e9b-4dbc-a9b5-e6db799076dd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.916084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a99bd9-5e9b-4dbc-a9b5-e6db799076dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.926200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a99bd9-5e9b-4dbc-a9b5-e6db799076dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:16.927678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a99bd9-5e9b-4dbc-a9b5-e6db799076dd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:16.930805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-e0a99bd9-5e9b-4dbc-a9b5-e6db799076dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:16.944744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:16.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 406/819] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:39:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f60d9-d4ca-4d3b-b4cc-16828403f9e3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:16.947777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f60d9-d4ca-4d3b-b4cc-16828403f9e3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:16.948660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f60d9-d4ca-4d3b-b4cc-16828403f9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:16.970578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c76f60d9-d4ca-4d3b-b4cc-16828403f9e3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:16.972257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c76f60d9-d4ca-4d3b-b4cc-16828403f9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:17.021594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e765a5-5901-401e-a127-50105e2a28b5 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.037022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:17.059719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.060770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.061143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.061520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/1a31e162778e4824b7bafd5dcd9b4b4b/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.063606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=1a31e162778e4824b7bafd5dcd9b4b4b, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.065176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:17.071350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 407/820] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:17.075654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f60d9-d4ca-4d3b-b4cc-16828403f9e3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:17.099557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 414/821] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:39:16 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/1a31e162778e4824b7bafd5dcd9b4b4b/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:39:17.102255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:17.280012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f60d9-d4ca-4d3b-b4cc-16828403f9e3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.312012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:17.321151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.322194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.322683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.323153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.324507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:17.325023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.325882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e7b2-375e-43b7-bea5-7ab0234d6fd8 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.326880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:17.347884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 408/822] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:39:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:17.353599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:17.367991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.368986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.369280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.369562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] PATH_INFO: `/v3/services/network` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.370437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.373062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:17.388884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:17.389915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:17.391212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 415/823] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:17 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:17.398867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:17.451962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:17.470922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bcb8f-305d-409e-8d94-b2e41bf110e4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.507755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.508939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.509511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.509994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.527686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1e857-e50c-4ed5-938e-991c7fa9f9ef None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.571449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.571787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:17.572050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.572340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.572623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.574245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] RBAC: Adding query filter params (name=network) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:17.574561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.576297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:17.582818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 416/824] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:17.588269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:17.591332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.593286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:17.711209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 409/825] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:17.716979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:17.779676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017095d3-f100-46f9-8f56-2328889ead1a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.781293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:17.823577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f4d1d-4533-4c61-a706-1c7f81f72a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.824519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f4d1d-4533-4c61-a706-1c7f81f72a70 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.824810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f4d1d-4533-4c61-a706-1c7f81f72a70 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.825089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f4d1d-4533-4c61-a706-1c7f81f72a70 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.825812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f4d1d-4533-4c61-a706-1c7f81f72a70 None admin] RBAC: Adding query filter params (type=network) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:17.826121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f4d1d-4533-4c61-a706-1c7f81f72a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.827644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f4d1d-4533-4c61-a706-1c7f81f72a70 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:17.833402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 417/826] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:17.838986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:17.840254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e408676-2d29-456e-9cb9-32880fcb9dac None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.841221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e408676-2d29-456e-9cb9-32880fcb9dac None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.841574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e408676-2d29-456e-9cb9-32880fcb9dac None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.843166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 418/827] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:17.843964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e408676-2d29-456e-9cb9-32880fcb9dac None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:17.854791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba360a87-6c9b-4c2d-a5b5-83ef88dd45b5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.890317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc02e1-1aa5-416b-9bac-5f439d22659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.891222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc02e1-1aa5-416b-9bac-5f439d22659b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.891573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc02e1-1aa5-416b-9bac-5f439d22659b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.891850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc02e1-1aa5-416b-9bac-5f439d22659b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.903127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc02e1-1aa5-416b-9bac-5f439d22659b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.904669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc02e1-1aa5-416b-9bac-5f439d22659b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-57dc02e1-1aa5-416b-9bac-5f439d22659b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:17.923671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:17.924582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:17.926253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:17.926253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:17.926253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:17.926253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:17.926253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 410/828] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:17.926884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f374-0521-4565-bc2b-bb556fa380b1 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.927025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f374-0521-4565-bc2b-bb556fa380b1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.928739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f374-0521-4565-bc2b-bb556fa380b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.944713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-bf89f374-0521-4565-bc2b-bb556fa380b1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:17.946095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bf89f374-0521-4565-bc2b-bb556fa380b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:17.952941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e408676-2d29-456e-9cb9-32880fcb9dac None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:17.989494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:17.990168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:17.990454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:17.990842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:17.992031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:17.993520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.023270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 419/829] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:18.024315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.031737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 411/830] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:18.052462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89f374-0521-4565-bc2b-bb556fa380b1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.075340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:18.128628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:18.169502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89f374-0521-4565-bc2b-bb556fa380b1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.180523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1799c9a-2333-4b78-be9b-c016cc7ae073 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.207288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.208448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.208835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.209184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.219900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.220581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.220862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.221087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:18.221191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.221855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:18.222163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:18.223606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:18.229706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:18.230736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:39:18.231604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:18.235555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 412/831] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:18.237334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.237561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 420/832] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:18.244418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.381134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:18.401198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f799c4-33df-4428-b858-11b85aa18d60 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.402043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e9b81-e7f7-4e9f-afb5-89f1b48ce03b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.438930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.439892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.440346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.440763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.441471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.441840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:18.442282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.442411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:18.442562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.442833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.444259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.450489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:18.451952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.453819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 413/833] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:18.458416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:18.459748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:18.470351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:18.471572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:18.472589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:18.473112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 421/834] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:18.484458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.582664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7fb2e-5b29-4194-bbef-e9a5579995e1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.616689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c43e6-218a-4d2e-a9eb-abc5e8eb07fe None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.627344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.628644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.628933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.629213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.648442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:18.650393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.656448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.657307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.657733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.658149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.659667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:18.660139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:18.662483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:18.671685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:18.672701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:18.673609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:18.673609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:18.673609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:18.673609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:18.675448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 414/835] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:18.680589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.682354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 422/836] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:18.688044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.707716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:18.731497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:18.805599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb733-7cdf-492a-94b7-dbbbe82e5f74 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.818159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948608ea-5548-4ff3-b7ee-113b777f0d38 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:18.846456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.847220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.847770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.848119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.849230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:18.849608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:18.851519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.861473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:18.862387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:18.863351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:18.863835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:18.865126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=abaa59b4e88642c681e9563295f6fad9) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:18.865619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:18.867226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:18.871456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 415/837] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:18.873579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:18.877787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 423/838] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/role_assignments?user.id=abaa59b4e88642c681e9563295f6fad9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:18.988770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:18.992491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6274d43-9146-4967-a94c-47c1e77f9347 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.030983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e2355-80f2-4a8a-a087-cd0f228e3e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:19.031868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e2355-80f2-4a8a-a087-cd0f228e3e28 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.032151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e2355-80f2-4a8a-a087-cd0f228e3e28 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.032426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e2355-80f2-4a8a-a087-cd0f228e3e28 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.040671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e2355-80f2-4a8a-a087-cd0f228e3e28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:19.042693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e2355-80f2-4a8a-a087-cd0f228e3e28 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-927e2355-80f2-4a8a-a087-cd0f228e3e28 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:19.058864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:19.059911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:19.060829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 416/839] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:39:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:19.067555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.145891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:19.183640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da58d5-9887-449e-8e61-373e9a64840d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.221632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:19.222322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.222607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.222887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.223777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:19.224092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:19.225515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:19.244769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1289a-55a6-414a-8500-0bc9ab80598b None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.244957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 424/840] 10.222.0.87 () {56 vars in 1149 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:19.245055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1289a-55a6-414a-8500-0bc9ab80598b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.245389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1289a-55a6-414a-8500-0bc9ab80598b None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.246926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 417/841] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:19.251400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.251754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8c412-8348-4e29-9b23-8910c199dc01 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.252052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8c412-8348-4e29-9b23-8910c199dc01 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.252340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8c412-8348-4e29-9b23-8910c199dc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.270734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-65b8c412-8348-4e29-9b23-8910c199dc01 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:19.271952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-65b8c412-8348-4e29-9b23-8910c199dc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:19.364101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdc43c-a78e-49d1-bd4f-37d961c859fa None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.369637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 418/842] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:19.392773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8c412-8348-4e29-9b23-8910c199dc01 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.395594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:19.406781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5c5d8-2a33-4f20-a950-0502ae4bdebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:19.407525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5c5d8-2a33-4f20-a950-0502ae4bdebe None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.407813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5c5d8-2a33-4f20-a950-0502ae4bdebe None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.408091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5c5d8-2a33-4f20-a950-0502ae4bdebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.480879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5c5d8-2a33-4f20-a950-0502ae4bdebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:19.482359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5c5d8-2a33-4f20-a950-0502ae4bdebe None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:19.497477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:19.511272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8c412-8348-4e29-9b23-8910c199dc01 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.552139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e530db7-6b57-4f66-8602-e48066a8aed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:19.553486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e530db7-6b57-4f66-8602-e48066a8aed8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.553966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e530db7-6b57-4f66-8602-e48066a8aed8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.554435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e530db7-6b57-4f66-8602-e48066a8aed8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.555294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e530db7-6b57-4f66-8602-e48066a8aed8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:19.556987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e530db7-6b57-4f66-8602-e48066a8aed8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-2e530db7-6b57-4f66-8602-e48066a8aed8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:19.569791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:19.570771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:19.571635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:19.574697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 419/843] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:19.577114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2529ed-f012-4959-ac5c-f802232bbc27 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.577397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2529ed-f012-4959-ac5c-f802232bbc27 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.577676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2529ed-f012-4959-ac5c-f802232bbc27 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.579060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 420/844] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:39:19.580546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2529ed-f012-4959-ac5c-f802232bbc27 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.593194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 425/845] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:19.594351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad3d1-8a2e-43a3-b755-9caca441d3cb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.594896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad3d1-8a2e-43a3-b755-9caca441d3cb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.594896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad3d1-8a2e-43a3-b755-9caca441d3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.611319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-e2dad3d1-8a2e-43a3-b755-9caca441d3cb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:19.612534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e2dad3d1-8a2e-43a3-b755-9caca441d3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:19.633738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:19.687231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2529ed-f012-4959-ac5c-f802232bbc27 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.688699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 426/846] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:19.692324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dad3d1-8a2e-43a3-b755-9caca441d3cb None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.725330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:19.726124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.726387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.726676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.727480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:19.727787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:19.729150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:19.738875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 421/847] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:19.738875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.757130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:19.800454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dad3d1-8a2e-43a3-b755-9caca441d3cb None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.835439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:19.836340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.836706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.837026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.844574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d13f23-ab83-41d2-82d0-9dded5db28bb None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.847866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:19.849393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:19.867113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:19.868030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:19.868885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:19.868885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:19.868885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:19.868885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:19.868885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 427/848] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:19.872235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.881933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:19.882825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:19.883343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:19.883615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:19.883897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:19.894736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:19.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:19.903593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:19.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 422/849] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:19.905087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:19.988631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42086cba-8a6f-453c-a2e4-b61df3942ff2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:19.989814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:20.013912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fbdbe-b2e9-48c4-854c-8bcdf0e455cd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.026397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.027498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.027779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.028043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.028765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:20.029080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.030464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.035264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 428/850] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:20.040658 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.052392 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.052909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.053299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.053803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.054670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:20.055043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.056580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.071037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 423/851] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:20.076997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.157073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:20.212778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b37ffa-2139-428a-a5cd-983b753178f4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.219097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9bb66-8ebc-48fd-a497-197f1d5f2087 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.252853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.253532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.253816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.254089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.254480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.254833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:20.255144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.255524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.255877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.256214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.256595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.263533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.265033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.266138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 429/852] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:20.267841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:20.271468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:20.282798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:20.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:20.284978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 424/853] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:20.291130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.400166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ff0a4-e1de-4605-a4c8-a9702965343a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.407185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb9b15-b3aa-445e-bd09-6c5c4f4eca37 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.433701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:20.441246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.442376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.442959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.443408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.456037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.456563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.457350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.457723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.458089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.458352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.459067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:20.459497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.461702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.479475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 425/854] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:20.481602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:20.482458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:20.483762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:20.483762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:20.483762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:20.483762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:20.483762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 430/855] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:20.485124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.488482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.566582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:20.598076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab93fd-6231-4a42-8ac3-0501414a42c0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.624173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520323-5b0c-4759-a654-8ac4dba7c2f8 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.633629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.634311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.634590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.634864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.635711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:20.635925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.637345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.654562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 431/856] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:20.655555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.675520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.676788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.677245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.677716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.678724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=cac0c2cb80a34d608aba65ba0c628f24) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:20.679710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.681604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.691428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 426/857] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/role_assignments?user.id=cac0c2cb80a34d608aba65ba0c628f24&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:20.711757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:20.776137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18d9b-d76a-4302-8adb-9cd3538792ae None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.817415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:20.818539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:20.818992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:20.819497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:20.828433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:20.830245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:20.847565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:20.848565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 12:39:20.849455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:20.851891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 432/858] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:20.857535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:20.965195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d885555-64a2-4c28-9016-570920bb1d34 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:20.989253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:21.003004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015d99b-18b0-4cd1-a841-113a6e0fd54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.003811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015d99b-18b0-4cd1-a841-113a6e0fd54d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.004160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015d99b-18b0-4cd1-a841-113a6e0fd54d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.004499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015d99b-18b0-4cd1-a841-113a6e0fd54d None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.005417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015d99b-18b0-4cd1-a841-113a6e0fd54d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:21.005792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015d99b-18b0-4cd1-a841-113a6e0fd54d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.007309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015d99b-18b0-4cd1-a841-113a6e0fd54d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.025797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 427/859] 10.222.0.87 () {56 vars in 1135 bytes} [Fri Jun 17 12:39:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:21.031255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:21.153373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b6ee8-d6d9-4a20-afde-f10d983e3c45 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:21.156395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:21.192624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.193289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.193569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.193839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.199588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6ff10-9c8b-40b7-beda-3ecd0727be4f None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.199885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6ff10-9c8b-40b7-beda-3ecd0727be4f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.200164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6ff10-9c8b-40b7-beda-3ecd0727be4f None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.201865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 428/860] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:21.207592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54873203-12e5-424c-b2a6-dc0849b0b5ca None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.207880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54873203-12e5-424c-b2a6-dc0849b0b5ca None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.208157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54873203-12e5-424c-b2a6-dc0849b0b5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.226179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-54873203-12e5-424c-b2a6-dc0849b0b5ca None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:21.227363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-54873203-12e5-424c-b2a6-dc0849b0b5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:21.264527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.266014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.299843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 429/861] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:21.324058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54873203-12e5-424c-b2a6-dc0849b0b5ca None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:21.392527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 433/862] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:21.393438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:21.397142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:21.446686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54873203-12e5-424c-b2a6-dc0849b0b5ca None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:21.462119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:21.494768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.495939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.496226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.496530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.497781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.499446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.500610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3f369-af04-469e-83b4-ce1baceff5ab None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:21.515696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:21.516659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:21.517476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:21.519547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 430/863] 10.222.0.87 () {56 vars in 1132 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:21.524451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:21.540376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.541184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.541465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.541793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.553493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.554949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:21.573993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:21.576151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:21.578255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:21.578255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:21.578255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:21.578255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:21.587394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 434/864] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:21.587394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:21.645897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:21.650597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6661-a819-4f23-b713-9347bc968e49 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:21.694144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00c424-2f86-4ee5-8b41-c2b3e43e2acf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:21.698231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.699552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.699981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.700423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.701283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:21.702241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.704345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.712134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 431/865] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:21.717679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:21.741395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.742796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.743522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.744017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.745162 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:21.746175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.748270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.761129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:21.766150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 435/866] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:21.771519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:21.832370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f951-1dd7-460b-a5a8-896b1c6867c8 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:21.875164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.876000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.876353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.876731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.877516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:21.877886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.879473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.889347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 432/867] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:21.908213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df888179-19c2-4663-bb5a-10136031cdc2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:21.928152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:21.949040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:21.949666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:21.949952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:21.950230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:21.958790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:21.960457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:21.976746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:21.977706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:21.979039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 436/868] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:21.984655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:22.098655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8c275-657f-4516-9a61-3e69a8570360 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:22.100350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:22.134767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098d8d3-4782-414e-85c9-e204e1989306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:22.135475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d8d3-4782-414e-85c9-e204e1989306 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:22.135764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d8d3-4782-414e-85c9-e204e1989306 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:22.136041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d8d3-4782-414e-85c9-e204e1989306 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:22.137156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d8d3-4782-414e-85c9-e204e1989306 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:22.137467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d8d3-4782-414e-85c9-e204e1989306 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:22.138948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d8d3-4782-414e-85c9-e204e1989306 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:22.154166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 433/869] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:22.160669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:22.217155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3b16b-1062-4484-95a6-ce280cb6ea31 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:22.217515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3b16b-1062-4484-95a6-ce280cb6ea31 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:22.217731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3b16b-1062-4484-95a6-ce280cb6ea31 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:22.219342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 434/870] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:22.224362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688d57c-10f1-400e-aa7e-d479b6335bdf None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:22.224614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688d57c-10f1-400e-aa7e-d479b6335bdf None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:22.224870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688d57c-10f1-400e-aa7e-d479b6335bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:22.246268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-f688d57c-10f1-400e-aa7e-d479b6335bdf None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:22.247238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f688d57c-10f1-400e-aa7e-d479b6335bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:22.277775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647308d4-2a57-4930-9747-e609f62524c4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:22.314126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:22.315112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:22.315602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:22.315886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:22.316712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=1a31e162778e4824b7bafd5dcd9b4b4b) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:22.317024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:22.318461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:22.331407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 437/871] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:22 2022] GET /identity/v3/role_assignments?user.id=1a31e162778e4824b7bafd5dcd9b4b4b&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:22.333782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 435/872] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:22.358024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:22.405591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:22.478946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5af90-a61e-4b18-af9e-e8f562f34f3b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:22.517330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083103f-c32e-45c4-ae0e-a740a68aead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:22.518017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083103f-c32e-45c4-ae0e-a740a68aead5 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:22.518318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083103f-c32e-45c4-ae0e-a740a68aead5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:22.518573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083103f-c32e-45c4-ae0e-a740a68aead5 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:22.519326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083103f-c32e-45c4-ae0e-a740a68aead5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:22.520735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083103f-c32e-45c4-ae0e-a740a68aead5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:22.537396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 438/873] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:22 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:23.025609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a63f3-6415-46e4-b3ec-a3a7164ef28a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:23.026681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a63f3-6415-46e4-b3ec-a3a7164ef28a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:23.027226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a63f3-6415-46e4-b3ec-a3a7164ef28a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:23.029107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 436/874] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:23.034663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2f5c1-e03e-4d8a-afe7-7c633ef92556 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:23.035187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2f5c1-e03e-4d8a-afe7-7c633ef92556 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:23.035690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2f5c1-e03e-4d8a-afe7-7c633ef92556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:23.053638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-cbf2f5c1-e03e-4d8a-afe7-7c633ef92556 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:23.055141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-cbf2f5c1-e03e-4d8a-afe7-7c633ef92556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:23.144391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 439/875] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:23 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:23.168983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a63f3-6415-46e4-b3ec-a3a7164ef28a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:23.263166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:23.274985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a63f3-6415-46e4-b3ec-a3a7164ef28a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:23.310523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:23.311354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:23.311655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:23.311929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:23.322454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:23.331034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:23.331993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:23.333231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 437/876] 10.222.0.87 () {56 vars in 1120 bytes} [Fri Jun 17 12:39:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:23.338459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2f5c1-e03e-4d8a-afe7-7c633ef92556 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:23.366432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:23.442583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2f5c1-e03e-4d8a-afe7-7c633ef92556 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:23.477186 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:23.478171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:23.478677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:23.479147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:23.480175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:23.480682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:23.482255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:23.492868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 440/877] 10.222.0.87 () {56 vars in 1115 bytes} [Fri Jun 17 12:39:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:23.499400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:23.602709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6bad0-7bba-4d2c-b641-f14d45dd752a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:23.637221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:23.638090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:23.638374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:23.638654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:23.649813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:23.651284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:23.670892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:23.671889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:23.673239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:23.673239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:23.673239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:23.673239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:23.673239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 438/878] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:23.678324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:23.776143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:23.790149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca8ad3-32fb-4cdb-9577-01481e4d47bd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:23.825929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:23.826603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:23.826885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:23.827161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:23.827948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:23.828253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:23.829612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:23.842408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 441/879] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:23.848024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:23.852167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:23.950855 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42138a-9c7e-4335-ad8b-b9c431e3a66e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:23.985146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315466d-eb92-4e9d-88a3-31284971e913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:23.986302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315466d-eb92-4e9d-88a3-31284971e913 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:23.986804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315466d-eb92-4e9d-88a3-31284971e913 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:23.987281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315466d-eb92-4e9d-88a3-31284971e913 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:23.996187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315466d-eb92-4e9d-88a3-31284971e913 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:23.997965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315466d-eb92-4e9d-88a3-31284971e913 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-4315466d-eb92-4e9d-88a3-31284971e913 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:24.013521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:24.014569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:24.016019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:24.018380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 439/880] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:39:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:24.024421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:24.136828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdc3e-5f84-4048-bbf1-edba9d823aef None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:24.145456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99db40-ad5b-4383-a9dc-d9f5006408ff None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.146000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99db40-ad5b-4383-a9dc-d9f5006408ff None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.146000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99db40-ad5b-4383-a9dc-d9f5006408ff None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.151123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 440/881] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:24.152361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f95d-494c-4359-bfc3-27979083475c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.152652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f95d-494c-4359-bfc3-27979083475c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.152937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f95d-494c-4359-bfc3-27979083475c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.170563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-79a0f95d-494c-4359-bfc3-27979083475c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:24.172055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-79a0f95d-494c-4359-bfc3-27979083475c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:24.173696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:24.174386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.174667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.175061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.175816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:24.176123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:24.177543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:24.247274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 441/882] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:24.247981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:24.270038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0f95d-494c-4359-bfc3-27979083475c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:24.322481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 442/883] 10.222.0.87 () {56 vars in 1149 bytes} [Fri Jun 17 12:39:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:24.328495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:24.337670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:24.382364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0f95d-494c-4359-bfc3-27979083475c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:24.422312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:24.422993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.423276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.423576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.424320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:24.425717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:24.443405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 442/884] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:24 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:39:24.457478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360fdc4-556e-411a-9f80-d31215808894 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:24.507811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cec6d-83f8-475a-9132-4bd9d17e7b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:24.508456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cec6d-83f8-475a-9132-4bd9d17e7b59 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.508733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cec6d-83f8-475a-9132-4bd9d17e7b59 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.509010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cec6d-83f8-475a-9132-4bd9d17e7b59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.588465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cec6d-83f8-475a-9132-4bd9d17e7b59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:24.590603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cec6d-83f8-475a-9132-4bd9d17e7b59 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:24.592819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:24.698052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 443/885] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:24 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:24.703824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:24.762035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03b577-efae-44f2-911f-ca6b116b165f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.762438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03b577-efae-44f2-911f-ca6b116b165f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.762816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03b577-efae-44f2-911f-ca6b116b165f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.769896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 444/886] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:24.770075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:24.770834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243dfd6-62e2-413c-98d3-fb822982ed21 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.771392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243dfd6-62e2-413c-98d3-fb822982ed21 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.773690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243dfd6-62e2-413c-98d3-fb822982ed21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.792000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3243dfd6-62e2-413c-98d3-fb822982ed21 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:24.794206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3243dfd6-62e2-413c-98d3-fb822982ed21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:24.819974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181af70-a24c-49b5-b3a1-b83be584725f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:24.854716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:24.855987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.856268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.856545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.865695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 445/887] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:24.867294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2da04-fd58-43f1-aa24-068c1901dba9 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.867605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2da04-fd58-43f1-aa24-068c1901dba9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.867889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2da04-fd58-43f1-aa24-068c1901dba9 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.869133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 446/888] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:24.876667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:24.877112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aab26-dbda-4ef3-8c78-333943353a99 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:24.877592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aab26-dbda-4ef3-8c78-333943353a99 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:24.878060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aab26-dbda-4ef3-8c78-333943353a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:24.878419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:24.896751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-935aab26-dbda-4ef3-8c78-333943353a99 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:24.897129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:24.897974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:24.898869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:24.898869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:24.898869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:24.898869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:24.899103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-935aab26-dbda-4ef3-8c78-333943353a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:24.899185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 443/889] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:24.899871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:24.931840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:24.975595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 447/890] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:24.976871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aab26-dbda-4ef3-8c78-333943353a99 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.029610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea940cf-100e-4a59-825f-a09499400100 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.036775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:25.066296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.067494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.067494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.067712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.068371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.069760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.079771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aab26-dbda-4ef3-8c78-333943353a99 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:25.082015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:25.083043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:25.083910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 444/891] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:25.084691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.121774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.122459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.122743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.123018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.123788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.124096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.125464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.141411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 448/892] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:25.141411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.164700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:25.206770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc636-96ad-49f2-a75c-476877abca0d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.241984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.242875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.243160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.243470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.243666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1671-8f32-407d-b697-28926cd98231 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.244127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.245593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:25.258065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:25.259109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:25.260202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 445/893] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:25.260996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.283252 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.284239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.284686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.285229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.286214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.286512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.287910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.297927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 449/894] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:25.297927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.307337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:25.377975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef618-7aac-42c8-a092-f4f79009f8bf None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.392048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:25.409089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49239b54-8fe6-44db-bfa4-294c449afcaf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.413532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.414348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.414633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.414911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.422276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.423758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:25.438102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:25.438988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:25.440012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 446/895] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:25.441391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.448450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165cca9-892b-480d-b092-68576c860288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.449180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165cca9-892b-480d-b092-68576c860288 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.449485 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165cca9-892b-480d-b092-68576c860288 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.449827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165cca9-892b-480d-b092-68576c860288 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.450542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165cca9-892b-480d-b092-68576c860288 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.451029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165cca9-892b-480d-b092-68576c860288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.452466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165cca9-892b-480d-b092-68576c860288 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.457242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 450/896] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:25.457805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165cca9-892b-480d-b092-68576c860288 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.545107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:25.552247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e491b-972b-41a7-b9cd-df9a61456c42 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.566794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165cca9-892b-480d-b092-68576c860288 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.589367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.590074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.590359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.590633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.591354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.591689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.593074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.606121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 447/897] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:25.606121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.619570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.620262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.620551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.620880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.622271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.622584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.624402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.642757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 451/898] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:25.648597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.667111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:25.720634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64988002-0f51-483d-9b32-062d08eb3ca0 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.760768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.761953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.762437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.762926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.764061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.764584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.766518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.773667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 448/899] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:25.779317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:25.793679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61128bb7-1cbe-41e9-bfb9-65e1bb186cdf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.811452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:25.838881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.839662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.840034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.840249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.841180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] RBAC: Adding query filter params (role.id=9357b61d1a7c4acbb490a56dd78c1784, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=cac0c2cb80a34d608aba65ba0c628f24) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.841492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.842928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.849296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 452/900] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/role_assignments?user.id=cac0c2cb80a34d608aba65ba0c628f24&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=9357b61d1a7c4acbb490a56dd78c1784 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:25.907851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a426-dccb-433f-a543-48ac2c60d4bf None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:25.950208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7c5e-f8d1-4b11-8ff6-4587c33c5983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:25.951424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7c5e-f8d1-4b11-8ff6-4587c33c5983 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:25.951924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7c5e-f8d1-4b11-8ff6-4587c33c5983 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:25.952393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7c5e-f8d1-4b11-8ff6-4587c33c5983 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:25.953461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7c5e-f8d1-4b11-8ff6-4587c33c5983 None admin] RBAC: Adding query filter params (interface=public, service_id=d34800996eca4bf1a71c285acf5f1948, region_id=RegionOne) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:25.953975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7c5e-f8d1-4b11-8ff6-4587c33c5983 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:25.955650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7c5e-f8d1-4b11-8ff6-4587c33c5983 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:25.962405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 449/901] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/endpoints?service_id=d34800996eca4bf1a71c285acf5f1948&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:25.968519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:26.059828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:26.082182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51681433-4505-4a7c-8946-b3cbd9ecab87 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:26.118669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553317b-d6c1-4182-a0c4-167581e682bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:26.119413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553317b-d6c1-4182-a0c4-167581e682bf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:26.119711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553317b-d6c1-4182-a0c4-167581e682bf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:26.119994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553317b-d6c1-4182-a0c4-167581e682bf None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:26.120721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553317b-d6c1-4182-a0c4-167581e682bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:26.122152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553317b-d6c1-4182-a0c4-167581e682bf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:26.128645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 453/902] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:25 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:39:26.641952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc578b-a0b3-4a14-9696-97dba225bfce None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:26.642516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc578b-a0b3-4a14-9696-97dba225bfce None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:26.642516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc578b-a0b3-4a14-9696-97dba225bfce None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:26.644166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 450/903] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:26.649694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541ab7-3460-4053-b160-139c98e08482 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:26.650126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541ab7-3460-4053-b160-139c98e08482 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:26.650306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541ab7-3460-4053-b160-139c98e08482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:26.668374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a5541ab7-3460-4053-b160-139c98e08482 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:26.669570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a5541ab7-3460-4053-b160-139c98e08482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:26.734444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 454/904] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:26.757474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc578b-a0b3-4a14-9696-97dba225bfce None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:26.852769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:26.869108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc578b-a0b3-4a14-9696-97dba225bfce None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:26.905070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:26.905861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:26.906148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:26.906425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:26.907105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:26.908705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:26.921968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:26.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:26.924410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 451/905] 10.222.0.87 () {56 vars in 1132 bytes} [Fri Jun 17 12:39:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:26.929196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5541ab7-3460-4053-b160-139c98e08482 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:26.997350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:27.030871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5541ab7-3460-4053-b160-139c98e08482 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:27.065380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:27.066055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:27.066340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:27.066618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:27.067337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:27.067678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:27.069049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:27.074243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 455/906] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:27.079966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:27.198216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e457-9465-48ea-82a7-7d8fadf94005 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:27.231665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:27.232334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:27.232615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:27.232889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:27.233605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:27.233913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:27.235293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:27.240523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 452/907] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:27.246685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:27.358686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa18eb-12e4-4bec-85ac-217c93682409 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:27.388971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:27.394251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901463c-d060-464c-9c0a-dc7c0645333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:27.395023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901463c-d060-464c-9c0a-dc7c0645333b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:27.395240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901463c-d060-464c-9c0a-dc7c0645333b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:27.395846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901463c-d060-464c-9c0a-dc7c0645333b None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:27.397100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901463c-d060-464c-9c0a-dc7c0645333b None admin] RBAC: Adding query filter params (interface=public, service_id=806cf16b05b24aba8d5062872bd61c9d, region_id=RegionOne) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:27.397581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901463c-d060-464c-9c0a-dc7c0645333b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:27.399058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901463c-d060-464c-9c0a-dc7c0645333b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:27.405159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 456/908] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:39:27 2022] GET /identity/v3/endpoints?service_id=806cf16b05b24aba8d5062872bd61c9d&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:27.410683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:27.428473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:27.517097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7cc15-e0cc-4e00-9f55-3c9bef3cf0e4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:27.552105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59bba1-145a-4df3-893d-4fae5a00ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:27.553160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59bba1-145a-4df3-893d-4fae5a00ae3c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:27.553653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59bba1-145a-4df3-893d-4fae5a00ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:27.554135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59bba1-145a-4df3-893d-4fae5a00ae3c None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:27.555050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59bba1-145a-4df3-893d-4fae5a00ae3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:27.556672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59bba1-145a-4df3-893d-4fae5a00ae3c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:27.563272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 453/909] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:27 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:39:27.968946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdbf6a-d9c3-4f0d-acdb-9ada31a854a9 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:27.969479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdbf6a-d9c3-4f0d-acdb-9ada31a854a9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:27.969479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdbf6a-d9c3-4f0d-acdb-9ada31a854a9 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:27.971217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 457/910] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:27.977161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bca978-2469-434f-b90b-572051d57e3e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:27.977668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bca978-2469-434f-b90b-572051d57e3e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:27.977742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bca978-2469-434f-b90b-572051d57e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:27.995468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-f9bca978-2469-434f-b90b-572051d57e3e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:27.996796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f9bca978-2469-434f-b90b-572051d57e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:28.036889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd74b-17f7-48a1-a871-c5d8f1ddaa62 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.037205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd74b-17f7-48a1-a871-c5d8f1ddaa62 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.037445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd74b-17f7-48a1-a871-c5d8f1ddaa62 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.043107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 458/911] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:28.044845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5060b-d7f3-4b84-8998-b3a825caf40e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.045131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5060b-d7f3-4b84-8998-b3a825caf40e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.045404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5060b-d7f3-4b84-8998-b3a825caf40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.062222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ffc5060b-d7f3-4b84-8998-b3a825caf40e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:28.063568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ffc5060b-d7f3-4b84-8998-b3a825caf40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:28.072691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 454/912] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:28.095903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bca978-2469-434f-b90b-572051d57e3e None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.109720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:28.151724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 459/913] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:28.165831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5060b-d7f3-4b84-8998-b3a825caf40e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.228870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bca978-2469-434f-b90b-572051d57e3e None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:28.267763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f1dfc-5c1a-4806-abce-de82d4b0b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:28.268460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1dfc-5c1a-4806-abce-de82d4b0b30c None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.268744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1dfc-5c1a-4806-abce-de82d4b0b30c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.269018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f1dfc-5c1a-4806-abce-de82d4b0b30c None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.269774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f1dfc-5c1a-4806-abce-de82d4b0b30c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:28.271400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f1dfc-5c1a-4806-abce-de82d4b0b30c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:28.290687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5060b-d7f3-4b84-8998-b3a825caf40e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:28.295730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 455/914] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:28 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:39:28.325023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:28.331145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7227ef-9982-4c2a-b803-d298a40050c9 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.331957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7227ef-9982-4c2a-b803-d298a40050c9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.332502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7227ef-9982-4c2a-b803-d298a40050c9 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.334042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 456/915] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:28.340300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93ef07-903e-4d7c-9659-d7d5c0b81e49 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.341235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93ef07-903e-4d7c-9659-d7d5c0b81e49 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.341640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93ef07-903e-4d7c-9659-d7d5c0b81e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.342112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:28.343010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.343363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.343746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.356550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:28.360882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0b93ef07-903e-4d7c-9659-d7d5c0b81e49 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:28.362208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0b93ef07-903e-4d7c-9659-d7d5c0b81e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:28.365111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:28.366097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:28.368257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 460/916] 10.222.0.87 () {56 vars in 1120 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:28.373292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.455699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:28.463711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 457/917] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:28.490233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93ef07-903e-4d7c-9659-d7d5c0b81e49 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.513776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ebf8-2334-4d51-9bca-3eddf7ebdfe9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:28.557605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:28.558281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.558692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.559364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.560150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:28.560461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:28.562169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:28.570603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 461/918] 10.222.0.87 () {56 vars in 1115 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:28.577629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.613780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93ef07-903e-4d7c-9659-d7d5c0b81e49 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:28.636004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:28.650268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:28.651337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.651874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.652809 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.653814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:28.655500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:28.669148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:28.670418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 12:39:28.671417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:28.674220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 458/919] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:28.680128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.702515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde3a6b-41c4-466e-a2a1-e8567d2e9fd9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:28.744610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:28.745432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.745720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.746037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.755748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:28.760913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:28.763249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:28.787331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:28.788298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:28.789196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:28.789196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:28.789196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:28.789196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:28.791994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 462/920] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:28.797608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.810400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4151167-492c-488a-819a-ed7b2053b129 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:28.856097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:28.857325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.857657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.857933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.858646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:28.859001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:28.860872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:28.872089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 459/921] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:28.873328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.926546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10329a2-755b-4bd0-81ad-fca5da17794d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:28.928136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:28.961241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:28.961948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:28.962230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:28.962499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:28.963219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:28.963547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:28.965039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:28.978269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 463/922] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:28.984538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:28.994592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca9c8e-c8e1-42bf-9c23-44f286983ca4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:29.026414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:29.037191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:29.038120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.038405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.038682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.039417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:29.039882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:29.041577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:29.052446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 460/923] 10.222.0.87 () {56 vars in 1114 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:29.053545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:29.103201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ac6dc-dcbe-4b7b-9549-eb81f2399666 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:29.143511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:29.144772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.145325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.145916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.155076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:29.156943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:29.174458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:29.175464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:29.176559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:29.183266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e43b4-e89b-4f2e-9601-899ca05c57b0 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:29.183949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 464/924] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:39:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:29.185251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:29.204739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:29.221820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:29.222763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.223050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.223327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.224126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:29.225671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:29.255772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 461/925] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:39:29.308093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a21d07-780e-492e-8efb-153c16efb5b5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:29.353506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:29.354198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.354481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.354756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.355798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:29.356112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:29.357799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:29.378330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 465/926] 10.222.0.87 () {56 vars in 1149 bytes} [Fri Jun 17 12:39:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:29.384540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:29.437872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:29.501437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0dc75-45c1-422e-8b50-7e91c2599a9a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:29.539042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c422-d8e8-433e-88cb-6d9c99bb3fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:29.540331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c422-d8e8-433e-88cb-6d9c99bb3fdb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.540837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c422-d8e8-433e-88cb-6d9c99bb3fdb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.541314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c422-d8e8-433e-88cb-6d9c99bb3fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.554118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c422-d8e8-433e-88cb-6d9c99bb3fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:29.555969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c422-d8e8-433e-88cb-6d9c99bb3fdb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-0a77c422-d8e8-433e-88cb-6d9c99bb3fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:29.574365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:29.575285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:29.576215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:29.576215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:29.576215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:29.576215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:29.578544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 462/927] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:29.584526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:29.659730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:29.694656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf88d2d-c89f-4602-a5ee-11ea16f77991 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:29.731501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:29.732284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.732567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.732842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.733572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:29.733877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:29.735292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:29.747842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 466/928] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:29.752296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5f53-8515-4d21-a71b-beb04cb95ff0 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.752743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5f53-8515-4d21-a71b-beb04cb95ff0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.752939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:29.753205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5f53-8515-4d21-a71b-beb04cb95ff0 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.754898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 463/929] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:29.761765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48677066-dcd7-47ef-b23e-dbd5495833b3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.762286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48677066-dcd7-47ef-b23e-dbd5495833b3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.762719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48677066-dcd7-47ef-b23e-dbd5495833b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.784808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-48677066-dcd7-47ef-b23e-dbd5495833b3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:29.786437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-48677066-dcd7-47ef-b23e-dbd5495833b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:29.861262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615af4f-0f62-4f2d-99e7-ce651031a8fd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:29.883515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 464/930] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:29.895014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:29.901680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:29.902644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:29.902935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:29.903209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:29.907661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48677066-dcd7-47ef-b23e-dbd5495833b3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:29.911639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:29.913098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:29.928851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:29.930715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:29.932463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 467/931] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:29.937183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:29.947395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:30.037932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48677066-dcd7-47ef-b23e-dbd5495833b3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:30.060076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87e193-11c1-484a-b0c1-f043d76057fd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:30.084778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:30.085643 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.086502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.086791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.087791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:30.089853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:30.101391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:30.102068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.102352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.102632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.103466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:30.103783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:30.104881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:30.105927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 12:39:30.106914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:30.108624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:30.108790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 465/932] 10.222.0.87 () {56 vars in 1132 bytes} [Fri Jun 17 12:39:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:30.118186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:30.131471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 468/933] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:30.138722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:30.206542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:30.249569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:30.262787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bec76-997c-42a1-a1f4-d114496f0464 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:30.276065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e79ac6-e577-4ba0-b933-e73f5f337afb None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:30.311420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:30.312506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.312882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.313620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.314711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:30.315111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:30.316900 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:30.318695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:30.319889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.320388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.320865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/cac0c2cb80a34d608aba65ba0c628f24/roles/9357b61d1a7c4acbb490a56dd78c1784` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.323405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 466/934] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:30.328199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=cac0c2cb80a34d608aba65ba0c628f24, role_id=9357b61d1a7c4acbb490a56dd78c1784)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:30.328601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:30.330012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:30.351417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 469/935] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:39:30 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/cac0c2cb80a34d608aba65ba0c628f24/roles/9357b61d1a7c4acbb490a56dd78c1784 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:39:30.450808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441f7ff-6697-464d-8e6d-651f4221761f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:30.491840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a25aa0-2f39-46c7-bb66-7debe5e68448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:30.492534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a25aa0-2f39-46c7-bb66-7debe5e68448 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.492868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a25aa0-2f39-46c7-bb66-7debe5e68448 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.493145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a25aa0-2f39-46c7-bb66-7debe5e68448 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.493872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a25aa0-2f39-46c7-bb66-7debe5e68448 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:30.494180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a25aa0-2f39-46c7-bb66-7debe5e68448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:30.495685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a25aa0-2f39-46c7-bb66-7debe5e68448 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:30.506696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 467/936] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:30.508372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:30.626972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1308177-de0c-43d1-8a29-10ee5923c8c5 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:30.638739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:30.662139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5498dd-52ee-4946-a1de-9485b3b4056f None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.662520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5498dd-52ee-4946-a1de-9485b3b4056f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.662707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5498dd-52ee-4946-a1de-9485b3b4056f None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.664410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 468/937] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:30.664588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:30.665071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.665413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.665621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.666362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:30.668223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:30.670085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cc74b-1dce-448a-9d61-c0243e985981 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.670435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cc74b-1dce-448a-9d61-c0243e985981 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.670618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cc74b-1dce-448a-9d61-c0243e985981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.693044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 470/938] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:30.700757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-178cc74b-1dce-448a-9d61-c0243e985981 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:30.702096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-178cc74b-1dce-448a-9d61-c0243e985981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:30.719072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:30.793819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 469/939] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:30.816455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:30.928041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618859e-adcf-45f9-87bb-20679d76a12c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:30.963859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:30.964602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:30.965222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:30.965222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:30.965825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:30.967256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:30.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:30.982560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:30.984605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:30.986333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 471/940] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:30.991401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cc74b-1dce-448a-9d61-c0243e985981 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:31.088842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cc74b-1dce-448a-9d61-c0243e985981 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:31.124831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:31.125544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:31.125887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:31.126165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:31.126933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:31.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:31.128653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:31.138704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 470/941] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:31.140151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:31.173092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:31.247012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323420-e9f9-45e6-807a-b759148568e4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:31.281895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:31.282610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:31.282892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:31.283165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:31.283986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:31.284299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:31.285677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:31.295512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 472/942] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:31.298030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:31.327760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:31.401339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9bb0b-c971-4e4b-9fd3-55d17f9e1df9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:31.436007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e495-7751-4bc7-8645-fb845f7d755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:31.436682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e495-7751-4bc7-8645-fb845f7d755d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:31.436966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e495-7751-4bc7-8645-fb845f7d755d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:31.437250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e495-7751-4bc7-8645-fb845f7d755d None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:31.438056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e495-7751-4bc7-8645-fb845f7d755d None admin] RBAC: Adding query filter params (interface=public, service_id=a87f4cc692fd4baf8584dbb88b17137b, region_id=RegionOne) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:31.438371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e495-7751-4bc7-8645-fb845f7d755d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:31.439829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e495-7751-4bc7-8645-fb845f7d755d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:31.451448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 471/943] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:39:31 2022] GET /identity/v3/endpoints?service_id=a87f4cc692fd4baf8584dbb88b17137b&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:31.453136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:31.521704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70cd5-22cc-4939-99d9-2dd1c200dfb6 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:31.521893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70cd5-22cc-4939-99d9-2dd1c200dfb6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:31.522174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70cd5-22cc-4939-99d9-2dd1c200dfb6 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:31.523833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 472/944] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:31.528580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1430c-f164-4e11-b069-4bb0d6575789 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:31.528862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1430c-f164-4e11-b069-4bb0d6575789 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:31.529141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1430c-f164-4e11-b069-4bb0d6575789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:31.550885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d2a1430c-f164-4e11-b069-4bb0d6575789 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:31.552505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d2a1430c-f164-4e11-b069-4bb0d6575789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:31.560906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:31.570252 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31acd86-0784-486f-b7c2-12c23f30e43d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:31.615167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:31.616022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:31.616366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:31.616701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:31.617456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:31.619599 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:31.635674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 473/945] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:31 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:39:31.642474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 473/946] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:31.666410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:31.779414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40adb3-34fc-4e6c-93bb-bd73bb4a2311 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:31.816468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:31.817140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:31.817418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:31.817689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:31.818408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:31.819858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:31.821241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING keystone.server.flask.common [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 17 12:39:31.841794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.sql.core [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 12:39:31.841794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 12:39:31.841794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [parameters: {'id': '8c92f66be0884f4ca1620ca1b48cedc5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 12:39:31.841794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=97009) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c92f66be0884f4ca1620ca1b48cedc5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c92f66be0884f4ca1620ca1b48cedc5. Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application conn.query(q) Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 12:39:31.859165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result.read() Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 17 12:39:31.860142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application next(self.gen) Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application yield resource Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application next(self.gen) Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise self.value Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application session.commit() Jun 17 12:39:31.861526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.session.flush() Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application compat.raise_( Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 12:39:31.862597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 12:39:31.864008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application conn.query(q) Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result.read() Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 12:39:31.869517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [parameters: {'id': '8c92f66be0884f4ca1620ca1b48cedc5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:31.873448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:31.875338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c92f66be0884f4ca1620ca1b48cedc5. Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:31.879803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 474/947] 10.222.0.87 () {60 vars in 1140 bytes} [Fri Jun 17 12:39:31 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:31.880350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1430c-f164-4e11-b069-4bb0d6575789 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:31.880350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:31.986280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1430c-f164-4e11-b069-4bb0d6575789 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:32.034848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d551c9-6534-476e-be98-64e7b05700dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:32.036488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d551c9-6534-476e-be98-64e7b05700dc None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:32.037188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d551c9-6534-476e-be98-64e7b05700dc None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:32.037901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d551c9-6534-476e-be98-64e7b05700dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:32.047755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d551c9-6534-476e-be98-64e7b05700dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:32.050397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d551c9-6534-476e-be98-64e7b05700dc None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-92d551c9-6534-476e-be98-64e7b05700dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:32.069863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:32.070787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:32.071698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:32.074657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 474/948] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:32.080831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:32.100391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:32.200343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f704037-51a0-4dc8-9f93-26abbece3eb9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:32.236231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bc972-b8a1-41d6-bdf2-8709e68b8201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:32.237296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bc972-b8a1-41d6-bdf2-8709e68b8201 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:32.237776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bc972-b8a1-41d6-bdf2-8709e68b8201 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:32.238271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bc972-b8a1-41d6-bdf2-8709e68b8201 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:32.239251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bc972-b8a1-41d6-bdf2-8709e68b8201 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:32.239786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bc972-b8a1-41d6-bdf2-8709e68b8201 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:32.241605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bc972-b8a1-41d6-bdf2-8709e68b8201 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:32.266693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 475/949] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:32.646866 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642c496-493a-4b57-8fc1-63ca4430fc9d None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:32.647453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642c496-493a-4b57-8fc1-63ca4430fc9d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:32.647453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642c496-493a-4b57-8fc1-63ca4430fc9d None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:32.649062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 475/950] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:32.654586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d5494-a69f-40dc-a517-5a947d0293ed None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:32.654875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d5494-a69f-40dc-a517-5a947d0293ed None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:32.655155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d5494-a69f-40dc-a517-5a947d0293ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:32.673625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-af5d5494-a69f-40dc-a517-5a947d0293ed None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:32.675129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-af5d5494-a69f-40dc-a517-5a947d0293ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:32.747822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 476/951] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:32.771139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642c496-493a-4b57-8fc1-63ca4430fc9d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:32.841910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:32.887779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642c496-493a-4b57-8fc1-63ca4430fc9d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:32.923169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31e098-d9ab-4301-bf85-51b71c424387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:32.924392 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31e098-d9ab-4301-bf85-51b71c424387 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:32.924865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31e098-d9ab-4301-bf85-51b71c424387 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:32.925356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31e098-d9ab-4301-bf85-51b71c424387 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:32.938007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31e098-d9ab-4301-bf85-51b71c424387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-ad31e098-d9ab-4301-bf85-51b71c424387 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:32.946836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:32.947878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:32.948736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:32.952282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 476/952] 10.222.0.87 () {56 vars in 1120 bytes} [Fri Jun 17 12:39:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:32.958618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d5494-a69f-40dc-a517-5a947d0293ed None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:32.962533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:33.012171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cc447-a237-4fc9-ad6c-227580845a75 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.012612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cc447-a237-4fc9-ad6c-227580845a75 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.012982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cc447-a237-4fc9-ad6c-227580845a75 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.014712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 477/953] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:39:33.020449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da163b02-ec13-4872-8b1d-422f02de063d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.020836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da163b02-ec13-4872-8b1d-422f02de063d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.021206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da163b02-ec13-4872-8b1d-422f02de063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.038779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-da163b02-ec13-4872-8b1d-422f02de063d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:33.040241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-da163b02-ec13-4872-8b1d-422f02de063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:33.064523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d5494-a69f-40dc-a517-5a947d0293ed None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.100150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.100987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.101359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.101728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.102573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:33.102973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.104570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.118896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 477/954] 10.222.0.87 () {56 vars in 1115 bytes} [Fri Jun 17 12:39:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:33.124656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 478/955] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:33.125462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.149722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da163b02-ec13-4872-8b1d-422f02de063d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.201967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:33.222829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:33.241279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604b7b1-824e-40c3-87e1-88aead2553ac None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.274802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da163b02-ec13-4872-8b1d-422f02de063d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.277711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.278776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.279259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.279758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.291809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.294078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.309958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.310764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.311046 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.311321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] PATH_INFO: `/v3/services/compute` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.312022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.313388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:33.314036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:33.315026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:33.316025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:33.316025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:33.316025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:33.316025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:33.322990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 478/956] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:33.324115 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:33.325484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:33.326605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:33.327520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 479/957] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:33.336234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.435112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a93bc-e907-48aa-bff2-dd7ddac136a2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.440091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92b84f-5091-4262-bcea-e49861392c1d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.454543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:33.458797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:33.476179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.476859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.477147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.477425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.478157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:33.478469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.479885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.482181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.483114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.483652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.484141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.485053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] RBAC: Adding query filter params (name=compute) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:33.485558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.487142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.492501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 479/958] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:33.493047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 480/959] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:33.498363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.500466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.606987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b62d-6fcb-4bf9-8d41-3d7460d7bbbc None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.615255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ef802-3f07-4911-aaef-3f84fc349fdd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.643359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.644102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.644390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.644669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.645486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] RBAC: Adding query filter params (type=compute) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:33.645801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.647172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.659510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 480/960] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:33.661051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.661905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.662188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.662461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.672187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.673695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:33.690314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:33.691327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:33.692268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 481/961] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:33.698260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.750321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:33.814007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604632d2-1551-4396-a296-5a735c0f2a86 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:33.855045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754f6c0-9332-4c1f-b945-eff9482edcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:33.855826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754f6c0-9332-4c1f-b945-eff9482edcf7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.856116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754f6c0-9332-4c1f-b945-eff9482edcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.856388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754f6c0-9332-4c1f-b945-eff9482edcf7 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.857354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754f6c0-9332-4c1f-b945-eff9482edcf7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:33.857666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754f6c0-9332-4c1f-b945-eff9482edcf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:33.859095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754f6c0-9332-4c1f-b945-eff9482edcf7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:33.878433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 481/962] 10.222.0.87 () {56 vars in 1149 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:33.884203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:33.891047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcacd7-6693-4a35-b7d7-26a7bf989633 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.891542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcacd7-6693-4a35-b7d7-26a7bf989633 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.891831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcacd7-6693-4a35-b7d7-26a7bf989633 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.896978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 482/963] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:39:33.898439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d56829-64af-4c3f-a5b4-9f52a882f719 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:33.898732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d56829-64af-4c3f-a5b4-9f52a882f719 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:33.899011 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d56829-64af-4c3f-a5b4-9f52a882f719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:33.916647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-10d56829-64af-4c3f-a5b4-9f52a882f719 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:33.918277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-10d56829-64af-4c3f-a5b4-9f52a882f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:33.933526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:33.995393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 483/964] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:34.007454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0c847-064e-4500-9c19-54a61d300650 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.016355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d56829-64af-4c3f-a5b4-9f52a882f719 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.050530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:34.066482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.067856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.068706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.069145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.140320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d56829-64af-4c3f-a5b4-9f52a882f719 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.156229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.158036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.186356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.188121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.188748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.189280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.190338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.192582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.203868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:34.213063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:34.214630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:34.215784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:34.223568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 484/965] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:34.225325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.277665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 482/966] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:33 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:34.279052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.344572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:34.353091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec194c8-5a88-4c4e-9192-a81e35f287cd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.388830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.389822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.390285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.390776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.391730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:34.392217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.393764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.399362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 485/967] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:34.404443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b48091-62ac-422f-8af2-6505699566e5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.405220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.442188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.443444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.443948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.444461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.455704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.457710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.460140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:34.479345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:34.480332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:34.481373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:34.481373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:34.481373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:34.481373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:34.488233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 483/968] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:34.489794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.528107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e72d5b-5aad-4c17-9746-d7939fbc42a2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.565098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.565994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.566106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.566349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.567064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:34.567391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.568771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.578859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 486/969] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:34.580303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.601723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6083a-3a16-462b-a9a8-8f7076b04687 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.605018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:34.638338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ddd44-d3b2-41e1-9901-0748fecff243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.639002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddd44-d3b2-41e1-9901-0748fecff243 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.639303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddd44-d3b2-41e1-9901-0748fecff243 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.639708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddd44-d3b2-41e1-9901-0748fecff243 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.640442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ddd44-d3b2-41e1-9901-0748fecff243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:34.640749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ddd44-d3b2-41e1-9901-0748fecff243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.642116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ddd44-d3b2-41e1-9901-0748fecff243 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.653780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 484/970] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:34.655115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92f9e7-4f6f-4282-a62c-b52b6da8da67 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.655433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92f9e7-4f6f-4282-a62c-b52b6da8da67 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.655719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92f9e7-4f6f-4282-a62c-b52b6da8da67 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.657330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 485/971] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:34.661509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92f9e7-4f6f-4282-a62c-b52b6da8da67 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.688732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0819f-b5ee-4df8-ba12-0dd0e0cf1113 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.721987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:34.724679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fd551-1fae-4d30-a389-fa263f1b77b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.725547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fd551-1fae-4d30-a389-fa263f1b77b8 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.726085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fd551-1fae-4d30-a389-fa263f1b77b8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.726461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fd551-1fae-4d30-a389-fa263f1b77b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.727288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fd551-1fae-4d30-a389-fa263f1b77b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.728760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fd551-1fae-4d30-a389-fa263f1b77b8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.749909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 487/972] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:34.759038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9d240-a29e-443a-902d-72cf9b20923a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.759420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9d240-a29e-443a-902d-72cf9b20923a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.759642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9d240-a29e-443a-902d-72cf9b20923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.780208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92f9e7-4f6f-4282-a62c-b52b6da8da67 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:34.792242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3fb9d240-a29e-443a-902d-72cf9b20923a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:34.795051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3fb9d240-a29e-443a-902d-72cf9b20923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:34.833685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:34.834960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:34.835533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:34.836059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:34.850694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:34.852906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:34.874490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:34.875853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:34.876963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:34.879531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 486/973] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:34.883424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 488/974] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:34.885134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.911012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9d240-a29e-443a-902d-72cf9b20923a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:34.923237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:35.011796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c934e-6d5d-4ec1-b743-9fe0d1aef141 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:35.023157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:35.034547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9d240-a29e-443a-902d-72cf9b20923a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:35.047515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:35.048324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:35.048723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:35.049091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:35.049952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:35.050324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:35.051848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:35.066581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 487/975] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:35.072598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:35.076146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:35.076922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:35.078031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:35.078031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:35.078271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:35.079833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:35.094153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.sql.core [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 12:39:35.094153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 12:39:35.094153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [parameters: {'user_id': '8fb87e0be37e492db796eb031ef29d7f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 12:39:35.094153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=97009) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application conn.query(q) Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result.read() Jun 17 12:39:35.102855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 12:39:35.103755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application next(self.gen) Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application yield resource Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application next(self.gen) Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise self.value Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application session.commit() Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 12:39:35.104729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.session.flush() Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application compat.raise_( Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 12:39:35.105681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 12:39:35.106569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application conn.query(q) Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application result.read() Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 12:39:35.107467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [parameters: {'user_id': '8fb87e0be37e492db796eb031ef29d7f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:35.108308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 17 12:39:35.109192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 17 12:39:35.110121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 17 12:39:35.110121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 17 12:39:35.110121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 12:39:35.110121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 12:39:35.110121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 12:39:35.110121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:35.110121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 489/976] 10.222.0.87 () {60 vars in 1131 bytes} [Fri Jun 17 12:39:34 2022] POST /identity/v3/users => generated 160 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:35.112525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:35.190872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:35.194187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a248-9c8a-4e3b-be38-f36e830fc3f1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:35.224280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fa68e-8991-4026-aea7-e3541d6a049d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:35.233365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:35.234434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:35.234891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:35.235342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:35.236487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] RBAC: Adding query filter params (role.id=9357b61d1a7c4acbb490a56dd78c1784, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=cac0c2cb80a34d608aba65ba0c628f24) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:35.236983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:35.238726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:35.245781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 488/977] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:35 2022] GET /identity/v3/role_assignments?user.id=cac0c2cb80a34d608aba65ba0c628f24&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=9357b61d1a7c4acbb490a56dd78c1784 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:35.261095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d3e90-0687-4b79-bd5e-59114a653745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:35.262183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d3e90-0687-4b79-bd5e-59114a653745 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:35.262625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d3e90-0687-4b79-bd5e-59114a653745 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:35.262999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d3e90-0687-4b79-bd5e-59114a653745 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:35.272733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d3e90-0687-4b79-bd5e-59114a653745 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:35.274711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d3e90-0687-4b79-bd5e-59114a653745 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-719d3e90-0687-4b79-bd5e-59114a653745 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:35.291438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:35.292447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:35.293909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 490/978] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:35.299721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:35.341918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:35.414224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791e5d5-5cf2-425b-b6d4-c212b06b166b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:35.449774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171766e7-1575-4448-98fc-da3864ee26e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:35.450467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171766e7-1575-4448-98fc-da3864ee26e9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:35.450754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171766e7-1575-4448-98fc-da3864ee26e9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:35.451033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171766e7-1575-4448-98fc-da3864ee26e9 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:35.451892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171766e7-1575-4448-98fc-da3864ee26e9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:35.452205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171766e7-1575-4448-98fc-da3864ee26e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:35.453663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171766e7-1575-4448-98fc-da3864ee26e9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:35.494274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 489/979] 10.222.0.87 () {56 vars in 1101 bytes} [Fri Jun 17 12:39:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:35.868800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80893d70-a0e7-4210-a919-2fb8d078f042 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:35.869195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80893d70-a0e7-4210-a919-2fb8d078f042 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:35.869374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80893d70-a0e7-4210-a919-2fb8d078f042 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:35.874317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 491/980] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:35.879834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99ae55-f8c1-4db1-89c1-fe38a330e4fb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:35.880316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99ae55-f8c1-4db1-89c1-fe38a330e4fb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:35.880393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99ae55-f8c1-4db1-89c1-fe38a330e4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:35.898033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ce99ae55-f8c1-4db1-89c1-fe38a330e4fb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:35.899219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ce99ae55-f8c1-4db1-89c1-fe38a330e4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:35.972341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 490/981] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:35.995146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80893d70-a0e7-4210-a919-2fb8d078f042 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:36.101486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80893d70-a0e7-4210-a919-2fb8d078f042 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:36.138179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:36.145756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bf190-8606-46ff-bafa-d022fda1e84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:36.146437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bf190-8606-46ff-bafa-d022fda1e84c None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:36.146717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bf190-8606-46ff-bafa-d022fda1e84c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:36.146999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bf190-8606-46ff-bafa-d022fda1e84c None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:36.147774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bf190-8606-46ff-bafa-d022fda1e84c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:36.149151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bf190-8606-46ff-bafa-d022fda1e84c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:36.162918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 492/982] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:35 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:36.985892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799fb67-aa71-45a4-8100-f3aa896c906c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:36.986443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799fb67-aa71-45a4-8100-f3aa896c906c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:36.986443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799fb67-aa71-45a4-8100-f3aa896c906c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:36.988100 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 491/983] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:36.993454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6244df-0655-483b-9614-56781d6cbcb5 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:36.993919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6244df-0655-483b-9614-56781d6cbcb5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:36.994012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6244df-0655-483b-9614-56781d6cbcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.011706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ec6244df-0655-483b-9614-56781d6cbcb5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:37.013576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ec6244df-0655-483b-9614-56781d6cbcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:37.089473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 493/984] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:37.116568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799fb67-aa71-45a4-8100-f3aa896c906c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.143201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:37.232588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799fb67-aa71-45a4-8100-f3aa896c906c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:37.270848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4ab2a-1105-44d1-9e48-7cc90c6c938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:37.272041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ab2a-1105-44d1-9e48-7cc90c6c938d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.272503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ab2a-1105-44d1-9e48-7cc90c6c938d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.272972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ab2a-1105-44d1-9e48-7cc90c6c938d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.273927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4ab2a-1105-44d1-9e48-7cc90c6c938d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:37.275536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4ab2a-1105-44d1-9e48-7cc90c6c938d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-96a4ab2a-1105-44d1-9e48-7cc90c6c938d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:37.288317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:37.289302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:37.290118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:37.295732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 492/985] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:37.300044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6244df-0655-483b-9614-56781d6cbcb5 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.407646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6244df-0655-483b-9614-56781d6cbcb5 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:37.425727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:37.438953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbec02-5a03-4afb-9ade-35d4591383a7 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.439286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbec02-5a03-4afb-9ade-35d4591383a7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.439588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbec02-5a03-4afb-9ade-35d4591383a7 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.444317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 493/986] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:37.447552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d812-d2bc-4220-b468-4fb5dae5e8a5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.447850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d812-d2bc-4220-b468-4fb5dae5e8a5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.448131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d812-d2bc-4220-b468-4fb5dae5e8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.450026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:37.451165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.451664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.452205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.452930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:37.453239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:37.454610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:37.461130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 494/987] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:37.467568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.468582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d2b4d812-d2bc-4220-b468-4fb5dae5e8a5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:37.470379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d2b4d812-d2bc-4220-b468-4fb5dae5e8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:37.557180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 494/988] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:37.569487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d812-d2bc-4220-b468-4fb5dae5e8a5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.588355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:37.593579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0108aa3-d6dd-4ef4-8371-b99fab0bd0d8 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:37.632654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:37.634637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.635086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.635533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.636971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:37.637493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:37.639400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:37.648567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 495/989] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:37.654085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.687461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d812-d2bc-4220-b468-4fb5dae5e8a5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:37.700583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:37.723774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:37.724581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.724862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.725186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] PATH_INFO: `/v3/services/image` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.725965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:37.727505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:37.739871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:37.740863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:37.741694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 495/990] 10.222.0.87 () {56 vars in 1105 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:37.746999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.758244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f678dd-5180-461d-8a13-2901d9e5b064 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:37.794401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:37.795076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.795392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.795670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.796484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] RBAC: Adding query filter params (interface=admin, service_id=a87f4cc692fd4baf8584dbb88b17137b, region_id=RegionOne) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:37.796801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:37.798181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:37.808069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 496/991] 10.222.0.87 () {56 vars in 1249 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/endpoints?service_id=a87f4cc692fd4baf8584dbb88b17137b&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:37.809226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.841304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:37.864595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702b9ce-8eaa-4f3b-89b6-313ad7b62231 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:37.899629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:37.900307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.900594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.900871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.901599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] RBAC: Adding query filter params (name=image) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:37.901911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:37.903413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:37.909317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 496/992] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:37.914710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:37.921258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81dc97-7fca-418f-b626-8be316d0770f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:37.931695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:37.965467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254f1ee-6baf-49f2-9dd2-82e6591ec14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:37.966465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f1ee-6baf-49f2-9dd2-82e6591ec14c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:37.966750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f1ee-6baf-49f2-9dd2-82e6591ec14c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:37.967029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f1ee-6baf-49f2-9dd2-82e6591ec14c None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:37.967758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254f1ee-6baf-49f2-9dd2-82e6591ec14c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:37.969222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254f1ee-6baf-49f2-9dd2-82e6591ec14c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:37.983416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 497/993] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:39:38.030262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b32677-bc95-4e15-8f77-63d56abb8a4a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:38.068974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2af09-d99a-4f92-8f71-222f3e8517eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:38.069618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2af09-d99a-4f92-8f71-222f3e8517eb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:38.069910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2af09-d99a-4f92-8f71-222f3e8517eb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:38.070191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2af09-d99a-4f92-8f71-222f3e8517eb None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:38.070932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2af09-d99a-4f92-8f71-222f3e8517eb None admin] RBAC: Adding query filter params (type=image) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:38.071244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2af09-d99a-4f92-8f71-222f3e8517eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:38.072711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2af09-d99a-4f92-8f71-222f3e8517eb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:38.078203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 497/994] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:38.316662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640087ed-49ca-4b55-b37e-b20f985a1600 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:38.317201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640087ed-49ca-4b55-b37e-b20f985a1600 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:38.317201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640087ed-49ca-4b55-b37e-b20f985a1600 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:38.318745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 498/995] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:38.324087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f74bfb-aedb-4592-baba-b9c0e864a6b8 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:38.324319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f74bfb-aedb-4592-baba-b9c0e864a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:38.324748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f74bfb-aedb-4592-baba-b9c0e864a6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:38.342686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-97f74bfb-aedb-4592-baba-b9c0e864a6b8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:38.343896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-97f74bfb-aedb-4592-baba-b9c0e864a6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:38.346111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:38.415239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 498/996] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:38.438588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640087ed-49ca-4b55-b37e-b20f985a1600 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:38.545884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640087ed-49ca-4b55-b37e-b20f985a1600 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:38.582890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:38.584066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:38.584594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:38.585071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] PATH_INFO: `/v3/services/compute` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:38.585925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:38.587608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:38.600036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:38.601058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:38.601921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:38.604753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 499/997] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:38.610560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f74bfb-aedb-4592-baba-b9c0e864a6b8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:38.713295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f74bfb-aedb-4592-baba-b9c0e864a6b8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:38.746947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:38.747660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:38.747940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:38.748215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:38.748928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:38.749231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:38.750614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:38.755811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 499/998] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:38.761424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:38.801812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:38.868792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1c38d-e885-4c26-84c7-a90018aeda73 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:38.902211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:38.902989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:38.903274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:38.903589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:38.904312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] RBAC: Adding query filter params (type=compute) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:38.904627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:38.906046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:38.915635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 500/999] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:38.916912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:38.920322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:39.016950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc921a-46b6-4db0-b9aa-4c29655cdef1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:39.051841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af9935-6e51-4dc9-aeb7-7e4b451d4d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:39.052586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af9935-6e51-4dc9-aeb7-7e4b451d4d07 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:39.052790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af9935-6e51-4dc9-aeb7-7e4b451d4d07 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:39.053121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af9935-6e51-4dc9-aeb7-7e4b451d4d07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:39.053932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af9935-6e51-4dc9-aeb7-7e4b451d4d07 None admin] RBAC: Adding query filter params (interface=public, service_id=8e59b5a3bbaa47e3ab29d457e799adf3, region_id=RegionOne) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:39.054247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af9935-6e51-4dc9-aeb7-7e4b451d4d07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:39.055649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af9935-6e51-4dc9-aeb7-7e4b451d4d07 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:39.061262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 500/1000] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:39:38 2022] GET /identity/v3/endpoints?service_id=8e59b5a3bbaa47e3ab29d457e799adf3&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:39.066790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:39.179831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:39.183243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea53a83-1b43-484a-b575-acfc8f085b22 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:39.218707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0dc09-fef7-44e3-af98-79b46ec0405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:39.219399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0dc09-fef7-44e3-af98-79b46ec0405e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:39.219695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0dc09-fef7-44e3-af98-79b46ec0405e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:39.219965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0dc09-fef7-44e3-af98-79b46ec0405e None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:39.220655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0dc09-fef7-44e3-af98-79b46ec0405e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:39.222014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0dc09-fef7-44e3-af98-79b46ec0405e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:39.235400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 501/1001] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:39 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:39:40.133923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a16d65-0334-4c59-9d11-dd6f0ef0ead7 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.134433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a16d65-0334-4c59-9d11-dd6f0ef0ead7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.134433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a16d65-0334-4c59-9d11-dd6f0ef0ead7 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.136072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 501/1002] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:40.141403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0273-c8d4-495d-abe5-238c1a01f719 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.141846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0273-c8d4-495d-abe5-238c1a01f719 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.141921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0273-c8d4-495d-abe5-238c1a01f719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.159840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a91d0273-c8d4-495d-abe5-238c1a01f719 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:40.161036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a91d0273-c8d4-495d-abe5-238c1a01f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:40.237395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 502/1003] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:40.256454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa084646-b119-457c-859e-666b12ff9838 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.256829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa084646-b119-457c-859e-666b12ff9838 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.257180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa084646-b119-457c-859e-666b12ff9838 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.259428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 502/1004] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:40.260499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d0273-c8d4-495d-abe5-238c1a01f719 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:40.265255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397cfff-e27e-4bb0-a763-c82c9d8d22cb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.265540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397cfff-e27e-4bb0-a763-c82c9d8d22cb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.265819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397cfff-e27e-4bb0-a763-c82c9d8d22cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.285278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e397cfff-e27e-4bb0-a763-c82c9d8d22cb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:40.286467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e397cfff-e27e-4bb0-a763-c82c9d8d22cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:40.356776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:40.364105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:40.366552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 503/1005] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:40.377136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d0273-c8d4-495d-abe5-238c1a01f719 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:40.395048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397cfff-e27e-4bb0-a763-c82c9d8d22cb None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:40.423114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:40.424087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.424374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.424763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.425442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:40.426816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:40.440114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:40.441164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:40.447212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 503/1006] 10.222.0.87 () {56 vars in 1126 bytes} [Fri Jun 17 12:39:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:40.448157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:40.505866 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397cfff-e27e-4bb0-a763-c82c9d8d22cb None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:40.542070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:40.542847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.543129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.543944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.544877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:40.546265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:40.554066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcfd98-e2ee-40e8-9e5f-fa6e35f1f59c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:40.564483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 504/1007] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:40 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:39:40.603352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:40.604077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.604361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.604637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.605362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:40.605671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:40.607143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:40.617878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 504/1008] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:40.619067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:40.685371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:40.736116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212697d-55f0-4e33-a440-9072d5b15b83 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:40.770963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387a3cd-e896-4cb0-83ff-c80f55b96069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:40.771704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387a3cd-e896-4cb0-83ff-c80f55b96069 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.771990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387a3cd-e896-4cb0-83ff-c80f55b96069 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.772257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387a3cd-e896-4cb0-83ff-c80f55b96069 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.772997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387a3cd-e896-4cb0-83ff-c80f55b96069 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:40.773297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387a3cd-e896-4cb0-83ff-c80f55b96069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:40.774752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387a3cd-e896-4cb0-83ff-c80f55b96069 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:40.780559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 505/1009] 10.222.0.87 () {56 vars in 1122 bytes} [Fri Jun 17 12:39:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:40.786307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:40.807764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:40.891287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f62ab-768c-4f04-b660-0344a6faff52 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:40.931234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdf811-f686-43ae-acef-70b3d8192dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:40.931937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdf811-f686-43ae-acef-70b3d8192dc7 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:40.932221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdf811-f686-43ae-acef-70b3d8192dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:40.932499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdf811-f686-43ae-acef-70b3d8192dc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:40.933247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdf811-f686-43ae-acef-70b3d8192dc7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:40.934733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdf811-f686-43ae-acef-70b3d8192dc7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:40.951247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 505/1010] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:41.466345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca375127-ee1a-4a07-bb3a-cf2e3bb05965 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:41.466924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca375127-ee1a-4a07-bb3a-cf2e3bb05965 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:41.466924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca375127-ee1a-4a07-bb3a-cf2e3bb05965 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:41.468463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 506/1011] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:41.474138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9e8d8-9b23-46d4-8f08-a4cac1ccfd3a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:41.474610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9e8d8-9b23-46d4-8f08-a4cac1ccfd3a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:41.474715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9e8d8-9b23-46d4-8f08-a4cac1ccfd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:41.494473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-67f9e8d8-9b23-46d4-8f08-a4cac1ccfd3a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:41.496140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-67f9e8d8-9b23-46d4-8f08-a4cac1ccfd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:41.569427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 506/1012] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:41 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:41.592684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca375127-ee1a-4a07-bb3a-cf2e3bb05965 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:41.704556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca375127-ee1a-4a07-bb3a-cf2e3bb05965 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:41.724877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:41.740807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:41.741621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:41.741903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:41.742178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:41.742845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:41.744267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:41.756363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:41.757488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 12:39:41.758402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:41.762769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 507/1013] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:41.763665 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9e8d8-9b23-46d4-8f08-a4cac1ccfd3a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:41.766894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:41.869310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9e8d8-9b23-46d4-8f08-a4cac1ccfd3a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:41.905183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:41.905932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:41.906214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:41.906490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:41.907203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:41.907541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:41.908995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:41.915164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 507/1014] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:41.920245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:42.022973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec43a0-7b35-4b6c-8e75-64135d1bf9ad None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:42.058125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b7d84-23a7-48f3-a056-365a00b9d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:42.058800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7d84-23a7-48f3-a056-365a00b9d23b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:42.059117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7d84-23a7-48f3-a056-365a00b9d23b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:42.059475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7d84-23a7-48f3-a056-365a00b9d23b None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:42.060190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b7d84-23a7-48f3-a056-365a00b9d23b None admin] RBAC: Adding query filter params (type=compute) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:42.060501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b7d84-23a7-48f3-a056-365a00b9d23b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:42.061890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b7d84-23a7-48f3-a056-365a00b9d23b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:42.067331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 508/1015] 10.222.0.87 () {56 vars in 1112 bytes} [Fri Jun 17 12:39:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:42.073157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:42.148171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:42.183981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b874e-0813-4d17-975c-3be0db34cad7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:42.217886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8595d-3d79-4d02-88a3-9b08d617fcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:42.218541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8595d-3d79-4d02-88a3-9b08d617fcbb None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:42.218822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8595d-3d79-4d02-88a3-9b08d617fcbb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:42.219101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8595d-3d79-4d02-88a3-9b08d617fcbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:42.219892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8595d-3d79-4d02-88a3-9b08d617fcbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:42.221271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8595d-3d79-4d02-88a3-9b08d617fcbb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:42.240430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 508/1016] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:42.719440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca8ce4-1b1b-4d09-a653-53cde3545ed0 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:42.720021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca8ce4-1b1b-4d09-a653-53cde3545ed0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:42.720021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca8ce4-1b1b-4d09-a653-53cde3545ed0 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:42.721373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 509/1017] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:42.726849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb5380-9272-4cb6-bbce-d2342b58cbca None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:42.727273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb5380-9272-4cb6-bbce-d2342b58cbca None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:42.727356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb5380-9272-4cb6-bbce-d2342b58cbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:42.745006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-cccb5380-9272-4cb6-bbce-d2342b58cbca None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:42.746190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-cccb5380-9272-4cb6-bbce-d2342b58cbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:42.822166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 509/1018] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:42.845518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca8ce4-1b1b-4d09-a653-53cde3545ed0 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:42.897946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:42.951680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca8ce4-1b1b-4d09-a653-53cde3545ed0 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:42.985515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611de8e-36ad-4d46-becd-ccde4643fb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:42.986238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611de8e-36ad-4d46-becd-ccde4643fb56 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:42.986520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611de8e-36ad-4d46-becd-ccde4643fb56 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:42.986797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611de8e-36ad-4d46-becd-ccde4643fb56 None admin] PATH_INFO: `/v3/services/image` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:42.987481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611de8e-36ad-4d46-becd-ccde4643fb56 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:42.988859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611de8e-36ad-4d46-becd-ccde4643fb56 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-3611de8e-36ad-4d46-becd-ccde4643fb56 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:43.000619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:43.001644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:43.002474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:43.006979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 510/1019] 10.222.0.87 () {56 vars in 1105 bytes} [Fri Jun 17 12:39:42 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:43.007893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb5380-9272-4cb6-bbce-d2342b58cbca None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:43.062626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:43.126491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb5380-9272-4cb6-bbce-d2342b58cbca None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:43.127562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5767-9a9b-42cf-a3e7-31145249ec49 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.127840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5767-9a9b-42cf-a3e7-31145249ec49 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.128130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5767-9a9b-42cf-a3e7-31145249ec49 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.129518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 511/1020] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:43.134615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f5ee8-ba55-4566-b045-0ecfef2ec459 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.134899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f5ee8-ba55-4566-b045-0ecfef2ec459 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.135185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f5ee8-ba55-4566-b045-0ecfef2ec459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.151848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-f39f5ee8-ba55-4566-b045-0ecfef2ec459 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:43.153161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f39f5ee8-ba55-4566-b045-0ecfef2ec459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:43.166496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:43.167172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.167486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.167754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.168898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] RBAC: Adding query filter params (name=image) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:43.169212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:43.170695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:43.175905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 510/1021] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:43.181874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:43.227441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 512/1022] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:43.250981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f5ee8-ba55-4566-b045-0ecfef2ec459 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:43.292603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaef83-cf16-41dc-a481-69ac1d273cc6 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:43.307786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:43.308711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:43.332639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:43.333320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.333611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.333886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.334603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:43.334937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:43.336345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:43.341500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 511/1023] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:43.347729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:43.370242 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f5ee8-ba55-4566-b045-0ecfef2ec459 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:43.408042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:43.409168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.409633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.410077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.411599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:43.413255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:43.415502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 513/1024] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:43.422068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:43.457595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609bc35-92da-4918-aa88-391dd0af37d0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:43.495325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:43.496029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.496317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.496596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] PATH_INFO: `/v3/endpoints` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.497450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] RBAC: Adding query filter params (interface=public, service_id=f8ec5dd546ae433f98f50ad638b263c0, region_id=RegionOne) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:43.497759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:43.499406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:43.505078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 512/1025] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:39:43 2022] GET /identity/v3/endpoints?service_id=f8ec5dd546ae433f98f50ad638b263c0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:43.510776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:43.526069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352a18e-5eee-4e69-a376-4b647c243298 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:43.538194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:43.557724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:43.566470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d03e1-b0f8-45da-a2f5-4f50616f2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:43.567142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d03e1-b0f8-45da-a2f5-4f50616f2cec None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.567450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d03e1-b0f8-45da-a2f5-4f50616f2cec None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.567725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d03e1-b0f8-45da-a2f5-4f50616f2cec None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.568864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d03e1-b0f8-45da-a2f5-4f50616f2cec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:43.570333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d03e1-b0f8-45da-a2f5-4f50616f2cec None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:43.597200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 514/1026] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:39:43 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:43.627582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b136136-0b69-450a-8b7f-fea18e73907c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:43.668100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a668b-122b-4c7c-aa2e-ffa068e64df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:43.669249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a668b-122b-4c7c-aa2e-ffa068e64df0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:43.669530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a668b-122b-4c7c-aa2e-ffa068e64df0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:43.669808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a668b-122b-4c7c-aa2e-ffa068e64df0 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:43.670527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a668b-122b-4c7c-aa2e-ffa068e64df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:43.671979 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a668b-122b-4c7c-aa2e-ffa068e64df0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:43.679990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 513/1027] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:39:43 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:39:44.465975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f18e6-9a41-4200-a608-d02b5b30f481 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:44.466508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f18e6-9a41-4200-a608-d02b5b30f481 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:44.466508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f18e6-9a41-4200-a608-d02b5b30f481 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:44.467828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 515/1028] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:44.473562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daafb0-1d4b-45fd-befe-e85fb3e0b8fd None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:44.473984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daafb0-1d4b-45fd-befe-e85fb3e0b8fd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:44.474140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daafb0-1d4b-45fd-befe-e85fb3e0b8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:44.491236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d6daafb0-1d4b-45fd-befe-e85fb3e0b8fd None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:44.492453 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d6daafb0-1d4b-45fd-befe-e85fb3e0b8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:44.565235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 514/1029] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:44.588624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f18e6-9a41-4200-a608-d02b5b30f481 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:44.695117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f18e6-9a41-4200-a608-d02b5b30f481 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:44.728704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:44.729477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:44.729761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:44.730018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:44.740940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:44.749495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:44.750518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:44.751360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 516/1030] 10.222.0.87 () {56 vars in 1120 bytes} [Fri Jun 17 12:39:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:44.759100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daafb0-1d4b-45fd-befe-e85fb3e0b8fd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:44.801163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:44.870876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daafb0-1d4b-45fd-befe-e85fb3e0b8fd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:44.906226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:44.906917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:44.907206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:44.907515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:44.908277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:44.908587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:44.909953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:44.924335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 515/1031] 10.222.0.87 () {56 vars in 1115 bytes} [Fri Jun 17 12:39:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:44.925311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:44.949796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:45.025483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed5e89-586d-442a-bccf-e85f21aea379 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:45.058722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:45.059582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.059872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.060140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.070216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:45.071706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:45.089370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:45.090271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:45.091814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:45.091814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:45.091814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:45.091814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:45.091814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 517/1032] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:45.098314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:45.220984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:45.225590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34440882-90b3-4b99-8cb5-216bc58f3dcd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:45.262135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:45.262864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.263148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.263428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.264165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:45.264476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:45.265847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:45.278242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 516/1033] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:45.283921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:45.364588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:45.392105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683aaf4-932c-41b8-a1d1-3dff378e8370 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:45.426581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:45.427390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.427703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.427972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] PATH_INFO: `/v3/users/nova` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.435279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:45.436800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:45.451363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:45.452408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:45.458494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 518/1034] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:39:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:45.459107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:45.565483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab432-a269-4d5f-8638-815e8b54476c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:45.601605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c23f3-d72d-4dab-9a4e-a383d79ae59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:45.602286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c23f3-d72d-4dab-9a4e-a383d79ae59d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.602617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c23f3-d72d-4dab-9a4e-a383d79ae59d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.602931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c23f3-d72d-4dab-9a4e-a383d79ae59d None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.603807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c23f3-d72d-4dab-9a4e-a383d79ae59d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:45.604119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c23f3-d72d-4dab-9a4e-a383d79ae59d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:45.605553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c23f3-d72d-4dab-9a4e-a383d79ae59d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:45.624404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 517/1035] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:39:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:45.630081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:45.736348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ff75-ae25-444a-b63f-d4dff8ffa01a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:45.777021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:45.777706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.777997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.778541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.781619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:45.814086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989daba-f95f-4cc3-85b0-d74bc396b554 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.814416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989daba-f95f-4cc3-85b0-d74bc396b554 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.814644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989daba-f95f-4cc3-85b0-d74bc396b554 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.816237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 518/1036] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:39:45.822639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66506-73eb-4805-b545-5da056c8741f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.822941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66506-73eb-4805-b545-5da056c8741f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.823223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66506-73eb-4805-b545-5da056c8741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.839660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-20e66506-73eb-4805-b545-5da056c8741f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:45.840888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-20e66506-73eb-4805-b545-5da056c8741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:45.851829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:45.856407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:45.857930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:45.909404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 519/1037] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:45.910801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68255-361e-4bbe-946c-a51f08142140 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.911098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68255-361e-4bbe-946c-a51f08142140 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.911404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68255-361e-4bbe-946c-a51f08142140 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.913663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 520/1038] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:45.919408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14e986-8237-4f63-b2c2-60bcadc11caa None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:45.919710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14e986-8237-4f63-b2c2-60bcadc11caa None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:45.920140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14e986-8237-4f63-b2c2-60bcadc11caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:45.938627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ff14e986-8237-4f63-b2c2-60bcadc11caa None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:45.939868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ff14e986-8237-4f63-b2c2-60bcadc11caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:45.965484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 519/1039] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:45.967119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:45.999792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.011981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 521/1040] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:46.013083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14e986-8237-4f63-b2c2-60bcadc11caa None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.073894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc4e3-967a-4395-8dd9-d9b2cd582872 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.087575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.108526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.109327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.110062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.110337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] PATH_INFO: `/v3/services/image` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.110903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14e986-8237-4f63-b2c2-60bcadc11caa None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.111002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.112402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:46.126191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:46.127246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:46.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 520/1041] 10.222.0.87 () {56 vars in 1105 bytes} [Fri Jun 17 12:39:45 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:46.128958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.150583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.151404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.151689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.151962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.162056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.163528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:46.181140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:46.182084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:46.182982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:46.182982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:46.182982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:46.182982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:46.182982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 522/1042] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:46.183723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.228333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.232417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73138b-e555-4bd8-a641-49c9cbae706f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.266199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.267013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.267296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.267605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] PATH_INFO: `/v3/domains/default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.268798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.270276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.271941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 521/1043] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:46.272705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.284597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efc293-4341-40bb-9fb5-4d4e9dd7409b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.312834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.321514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.322183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.322463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.322737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.323482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] RBAC: Adding query filter params (name=image) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:46.323797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.325256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.330029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 523/1044] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:46.330789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.387799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985aabfc-e358-4c78-92c9-1b11b7e4098a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.423382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.424076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.424357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.424633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.425366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:46.425674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.427043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.431491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c873b-23e9-460a-b49b-f06f68c6998e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.439417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 522/1045] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:46.440008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.467837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.469896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.470766 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.471052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.471334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.472320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.473644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.493659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 524/1046] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:39:46 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:39:46.494685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.548192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc52bc-1d08-4501-9102-90386de64aa7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.560106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.588353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.588974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.589268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.589547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.590292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] RBAC: Adding query filter params (type=image) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:46.590717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.592406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.598086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 523/1047] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:46.603914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.632127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db195-c36f-456a-a9d5-3ac0097e3e5b None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.671174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.672041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.672324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.672600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.681175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.682641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:46.701180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:46.702265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:46.703177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:46.705070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 525/1048] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:46.711107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:46.720260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.732098 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10133649-d2a3-4cc9-8396-fecec6cbf639 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.768655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.770219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.770691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.771177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.772139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.774531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.800625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 524/1049] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:39:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:39:46.826654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64892f-4adb-42de-8040-6b10180c0871 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:46.838295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:46.874662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51fdb5-bf36-4608-a2aa-ce65f15780bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:46.875255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51fdb5-bf36-4608-a2aa-ce65f15780bf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:46.875572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51fdb5-bf36-4608-a2aa-ce65f15780bf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:46.875849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51fdb5-bf36-4608-a2aa-ce65f15780bf None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:46.876661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51fdb5-bf36-4608-a2aa-ce65f15780bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:46.876972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51fdb5-bf36-4608-a2aa-ce65f15780bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:46.878452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51fdb5-bf36-4608-a2aa-ce65f15780bf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:46.892142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 526/1050] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:46.898485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:47.012165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ade1f7-1ecb-4060-9c09-0a0e9e8f7702 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:47.050461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42c55f-b18f-4551-bc1f-231bfe2fe65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:47.051135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c55f-b18f-4551-bc1f-231bfe2fe65a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:47.051476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c55f-b18f-4551-bc1f-231bfe2fe65a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:47.051754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c55f-b18f-4551-bc1f-231bfe2fe65a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:47.052597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42c55f-b18f-4551-bc1f-231bfe2fe65a None admin] RBAC: Adding query filter params (role.id=9357b61d1a7c4acbb490a56dd78c1784, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=abaa59b4e88642c681e9563295f6fad9) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:47.052911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42c55f-b18f-4551-bc1f-231bfe2fe65a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:47.054356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42c55f-b18f-4551-bc1f-231bfe2fe65a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:47.060273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 525/1051] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:46 2022] GET /identity/v3/role_assignments?user.id=abaa59b4e88642c681e9563295f6fad9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=9357b61d1a7c4acbb490a56dd78c1784 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:48.674628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ce887-c0dc-404c-ab80-70e7ad637f5c None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:48.675182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ce887-c0dc-404c-ab80-70e7ad637f5c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:48.675182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ce887-c0dc-404c-ab80-70e7ad637f5c None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:48.676571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 527/1052] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:48.682175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916361-d62c-43f5-beb1-26d8bfc34d0b None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:48.682641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916361-d62c-43f5-beb1-26d8bfc34d0b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:48.682729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916361-d62c-43f5-beb1-26d8bfc34d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:48.699968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-f7916361-d62c-43f5-beb1-26d8bfc34d0b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:48.701204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f7916361-d62c-43f5-beb1-26d8bfc34d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:48.704987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:48.771105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 526/1053] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:48.796083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ce887-c0dc-404c-ab80-70e7ad637f5c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:48.901278 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ce887-c0dc-404c-ab80-70e7ad637f5c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:48.940816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:48.941538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:48.941815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:48.942106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:48.953134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:48.960730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:48.961635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:48.962490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 528/1054] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:48.968518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0889b-0d5e-4970-a2ef-f272e67cca00 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:48.968808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0889b-0d5e-4970-a2ef-f272e67cca00 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:48.969088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0889b-0d5e-4970-a2ef-f272e67cca00 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:48.970471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 527/1055] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:48.971346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:48.975184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6182f-837d-45b1-8fcf-a6600e46cbcc None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:48.975514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6182f-837d-45b1-8fcf-a6600e46cbcc None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:48.975780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6182f-837d-45b1-8fcf-a6600e46cbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:48.993931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-dbf6182f-837d-45b1-8fcf-a6600e46cbcc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:48.995114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-dbf6182f-837d-45b1-8fcf-a6600e46cbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:49.000028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.064758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 528/1056] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:49.080691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3060c-f0bb-4045-bdbf-9d55e7b7aafa None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.087009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6182f-837d-45b1-8fcf-a6600e46cbcc None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.124496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.125299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.125591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.125866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.126645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:49.126957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.128439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.137713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 529/1057] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:49.143878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.177041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.209158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6182f-837d-45b1-8fcf-a6600e46cbcc None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.231064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.242439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f045d3c-3011-4a43-bd3d-debe18a24c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.243109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045d3c-3011-4a43-bd3d-debe18a24c4a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.243412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045d3c-3011-4a43-bd3d-debe18a24c4a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.243698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045d3c-3011-4a43-bd3d-debe18a24c4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.245652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe5ddb-d63a-45fc-951b-945ec86c4ff2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.279535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8c275-747c-4fae-b4aa-61902911ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.280335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8c275-747c-4fae-b4aa-61902911ef92 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.280619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8c275-747c-4fae-b4aa-61902911ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.280892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8c275-747c-4fae-b4aa-61902911ef92 None admin] PATH_INFO: `/v3/users/dfd8e36e9cd4438297e23716269eb6a5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.295014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8c275-747c-4fae-b4aa-61902911ef92 None admin] RBAC: Authorizing `identity:get_user(user_id=dfd8e36e9cd4438297e23716269eb6a5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.296518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8c275-747c-4fae-b4aa-61902911ef92 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.298219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 530/1058] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/users/dfd8e36e9cd4438297e23716269eb6a5 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:49.299455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0d888-a0c2-4bcf-9d1e-7efc5cc62944 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.299747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0d888-a0c2-4bcf-9d1e-7efc5cc62944 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.300022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0d888-a0c2-4bcf-9d1e-7efc5cc62944 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.301454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 531/1059] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:49.303849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0d888-a0c2-4bcf-9d1e-7efc5cc62944 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.312875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f045d3c-3011-4a43-bd3d-debe18a24c4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.314345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f045d3c-3011-4a43-bd3d-debe18a24c4a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.382117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.408263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0d888-a0c2-4bcf-9d1e-7efc5cc62944 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.413370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 529/1060] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 12:39:49.414507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d537085-b978-4dcd-a155-4515f883987d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.414798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d537085-b978-4dcd-a155-4515f883987d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.415074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d537085-b978-4dcd-a155-4515f883987d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.431462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8d537085-b978-4dcd-a155-4515f883987d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:49.432647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8d537085-b978-4dcd-a155-4515f883987d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:49.447659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.448301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.448579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.448848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.464332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.499637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 530/1061] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:49.500477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d537085-b978-4dcd-a155-4515f883987d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.528193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.529877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.597550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.600871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d537085-b978-4dcd-a155-4515f883987d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.629344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 532/1062] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 12:39:49.630194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.639393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.640245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.640889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.641178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.652971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.654511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:49.671758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:49.672804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:49.673918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:49.673918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:49.673918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:49.673918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:49.673918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 531/1063] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:49.673918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.678653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.735598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b84ee-084c-4b0b-bc70-9db4841b0b18 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.768927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.769753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.770026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.770294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.775002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ec22e-4c42-497e-b825-3a959bd704f5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.780173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:49.788316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:49.789444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 533/1064] 10.222.0.87 () {56 vars in 1120 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:49.790315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.816112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.816994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.817276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.817568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.818725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] RBAC: Authorizing `identity:get_project(project_id=5984bda6cb484d85ac14b873bb27fe81)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.820222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.821929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 532/1065] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:49.822674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.843682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.895695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddbaa8-0427-447f-8e71-cfdbea9420cf None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.897682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:49.922156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d73cf-848b-4023-9453-1fd0c09980c5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:49.930606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.931269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.931595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.931871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.932593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:49.932902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.934270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.945884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 534/1066] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:49.946672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:49.962480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:49.963185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:49.963518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:49.963795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:49.964575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:49.964882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:49.966535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:49.982189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 533/1067] 10.222.0.87 () {56 vars in 1115 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:49.983576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:50.063427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dc1a2-e446-4949-a7fd-e06297bd26ee None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:50.074223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:50.094633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ca431-f8e9-4056-9b6b-ecef85b5a3fb None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:50.102815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:50.103524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:50.103815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:50.104089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:50.104935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=5984bda6cb484d85ac14b873bb27fe81, user.id=dfd8e36e9cd4438297e23716269eb6a5) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:50.105245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:50.106757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:50.115293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 535/1068] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/role_assignments?user.id=dfd8e36e9cd4438297e23716269eb6a5&scope.project.id=5984bda6cb484d85ac14b873bb27fe81&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:50.140493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:50.141339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:50.141618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:50.141890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:50.155990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:50.157817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:50.177418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:50.178406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:50.179312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:50.179312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:50.179312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:50.179312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:50.179312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 534/1069] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:50.184827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:50.218458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:50.303664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7e9cb-ae5a-4908-9614-de04e9bf8b1f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:50.337246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:50.337831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:50.338118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:50.338394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:50.339123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:50.339449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:50.340849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:50.353881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 536/1070] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:50.360024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:50.464719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a06af4-38a7-44d1-a04e-7a145029406b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:50.498316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:50.499116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:50.499397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:50.499693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:50.501674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:50.508948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:50.510628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:50.525431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:50.526417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:50.527810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 535/1071] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:39:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:50.533001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:50.625195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:50.638204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1294e3-816d-4057-999a-08a869ad54d9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:50.674469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:50.675149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:50.675451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:50.675709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:50.676549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:50.676854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:50.678288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:50.695648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 537/1072] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:39:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:50.701461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:50.808021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05ab27-39ed-4f37-b81c-16638f91bc22 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:50.841676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:50.842474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:50.842756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:50.843028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:50.853964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:50.855450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:50.872623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:50.873679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:50.874576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:50.874576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:50.874576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:50.874576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:50.874576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 536/1073] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:50.879750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:50.980094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc8d8a-3a62-4335-bba3-caa785f7fafc None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:51.015311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:51.015994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:51.016269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:51.016535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:51.017254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:51.017558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:51.018912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:51.030799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 538/1074] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:51.036604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:51.086634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:51.143427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756669ec-7449-47c0-b4a9-703fee89d328 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:51.177494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:51.178331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:51.178618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:51.178902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:51.186670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:51.188192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:51.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:51.203947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:51.204832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 537/1075] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:51.210038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:51.220443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:51.314047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859367ee-91b7-4303-8fa0-51bd51a8c444 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:51.349364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b5395-4f6b-42b1-8b06-85fdbfbc04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:51.350023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5395-4f6b-42b1-8b06-85fdbfbc04a9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:51.350304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5395-4f6b-42b1-8b06-85fdbfbc04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:51.350581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5395-4f6b-42b1-8b06-85fdbfbc04a9 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:51.351398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b5395-4f6b-42b1-8b06-85fdbfbc04a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:51.351709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b5395-4f6b-42b1-8b06-85fdbfbc04a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:51.353113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b5395-4f6b-42b1-8b06-85fdbfbc04a9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:51.365693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 539/1076] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:51.371811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:51.477995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3f494-783d-4574-91ec-1de07d54fd63 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:51.485552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:51.512875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57369aef-2864-4795-bf57-73e99d7d7c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:51.513990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57369aef-2864-4795-bf57-73e99d7d7c61 None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:51.514275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57369aef-2864-4795-bf57-73e99d7d7c61 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:51.514570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57369aef-2864-4795-bf57-73e99d7d7c61 None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/abaa59b4e88642c681e9563295f6fad9/roles/9357b61d1a7c4acbb490a56dd78c1784` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:51.516823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57369aef-2864-4795-bf57-73e99d7d7c61 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=abaa59b4e88642c681e9563295f6fad9, role_id=9357b61d1a7c4acbb490a56dd78c1784)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:51.518274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57369aef-2864-4795-bf57-73e99d7d7c61 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:51.531521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 538/1077] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:39:51 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/abaa59b4e88642c681e9563295f6fad9/roles/9357b61d1a7c4acbb490a56dd78c1784 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:39:52.204365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779130-e23d-4ed7-9918-a30b647aa3d4 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:52.204932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779130-e23d-4ed7-9918-a30b647aa3d4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:52.204932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82779130-e23d-4ed7-9918-a30b647aa3d4 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:52.206315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 540/1078] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:52.212075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcd4e-4f8f-437f-b277-e894c76d8cd5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:52.212464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcd4e-4f8f-437f-b277-e894c76d8cd5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:52.212636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcd4e-4f8f-437f-b277-e894c76d8cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:52.230826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-538fcd4e-4f8f-437f-b277-e894c76d8cd5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:52.232094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-538fcd4e-4f8f-437f-b277-e894c76d8cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:52.301931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 539/1079] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:52.325134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82779130-e23d-4ed7-9918-a30b647aa3d4 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:52.392230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:52.430450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82779130-e23d-4ed7-9918-a30b647aa3d4 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:52.465617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:52.466596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:52.466876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:52.467131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:52.477077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:52.485017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:52.485965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:52.487135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 541/1080] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:52.492540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcd4e-4f8f-437f-b277-e894c76d8cd5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:52.587405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:52.594901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcd4e-4f8f-437f-b277-e894c76d8cd5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:52.632919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:52.633597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:52.633888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:52.634156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:52.634910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:52.635217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:52.636628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:52.645094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 540/1081] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:52.650905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:52.753483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712e50-fd03-4a6a-9657-f79ac1458eae None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:52.778916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:52.788027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:52.788900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:52.789183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:52.789439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] PATH_INFO: `/v3/users/dfd8e36e9cd4438297e23716269eb6a5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:52.790623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] RBAC: Authorizing `identity:get_user(user_id=dfd8e36e9cd4438297e23716269eb6a5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:52.792100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:52.794467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 542/1082] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:39:52 2022] GET /identity/v3/users/dfd8e36e9cd4438297e23716269eb6a5 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:52.800681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:52.902339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5faf2-7b5e-4099-9ba5-9e38dcffeafd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:52.936650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc381f5-d104-4aa5-b24e-afefaf0f9f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:52.937516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc381f5-d104-4aa5-b24e-afefaf0f9f4b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:52.937797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc381f5-d104-4aa5-b24e-afefaf0f9f4b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:52.938067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc381f5-d104-4aa5-b24e-afefaf0f9f4b None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:52.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc381f5-d104-4aa5-b24e-afefaf0f9f4b None admin] RBAC: Authorizing `identity:get_project(project_id=5984bda6cb484d85ac14b873bb27fe81)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:52.940834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc381f5-d104-4aa5-b24e-afefaf0f9f4b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:52.942547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 541/1083] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:39:52 2022] GET /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:52.948727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:53.052622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e5e91-c5fb-4f83-a3b5-bf8d83630324 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:53.087689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efef35-cc63-4634-82ab-1b28b83b87f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:53.088607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efef35-cc63-4634-82ab-1b28b83b87f9 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:53.088890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efef35-cc63-4634-82ab-1b28b83b87f9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:53.089165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efef35-cc63-4634-82ab-1b28b83b87f9 None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81/users/dfd8e36e9cd4438297e23716269eb6a5/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:53.091319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efef35-cc63-4634-82ab-1b28b83b87f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5984bda6cb484d85ac14b873bb27fe81, user_id=dfd8e36e9cd4438297e23716269eb6a5, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:53.092796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efef35-cc63-4634-82ab-1b28b83b87f9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:53.107413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 543/1084] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:39:52 2022] PUT /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81/users/dfd8e36e9cd4438297e23716269eb6a5/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:39:53.624594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef21b0c-548b-4474-8cb7-017b659042d8 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:53.625215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef21b0c-548b-4474-8cb7-017b659042d8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:53.625215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef21b0c-548b-4474-8cb7-017b659042d8 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:53.626520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 542/1085] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:53.631869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046f33-2f49-4b44-a33e-0c7a195ece11 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:53.632292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046f33-2f49-4b44-a33e-0c7a195ece11 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:53.632497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046f33-2f49-4b44-a33e-0c7a195ece11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:53.650433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-21046f33-2f49-4b44-a33e-0c7a195ece11 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:53.651659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-21046f33-2f49-4b44-a33e-0c7a195ece11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:53.702555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:53.721857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 544/1086] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:53.747550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef21b0c-548b-4474-8cb7-017b659042d8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:53.776008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:53.854682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef21b0c-548b-4474-8cb7-017b659042d8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:53.889822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:53.890575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:53.890857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:53.891129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:53.902135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:53.910772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:53.911784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:53.913169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 543/1087] 10.222.0.87 () {56 vars in 1120 bytes} [Fri Jun 17 12:39:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:53.918240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046f33-2f49-4b44-a33e-0c7a195ece11 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:54.016493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046f33-2f49-4b44-a33e-0c7a195ece11 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:54.051843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:54.052511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:54.052793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:54.053070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:54.053832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:54.054142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:54.055552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:54.063689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 545/1088] 10.222.0.87 () {56 vars in 1115 bytes} [Fri Jun 17 12:39:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:54.069525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:54.162320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:54.179813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f63f68-d374-4503-9697-296c7fba0e66 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:54.214196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:54.214993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:54.215276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:54.215586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:54.226283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:54.227778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:54.245462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:54.246437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:54.247329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:54.247329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:54.247329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:54.247329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:54.247329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 544/1089] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:54.252711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:54.301786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:54.356649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527b1eb-1b62-412b-b001-0dafd449b136 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:54.390908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:54.391601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:54.391888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:54.392168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:54.392897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:54.393205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:54.394575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:54.408147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 546/1090] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:54.414048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:54.515114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfaca-664f-43ee-a437-2bf6b52e3377 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:54.550126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:54.550913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:54.551202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:54.551506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:54.558586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:54.560274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:54.562280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:39:54.576516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:54.582488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:54.583930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 545/1091] 10.222.0.87 () {56 vars in 1093 bytes} [Fri Jun 17 12:39:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:54.584591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:54.685839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f817de-8684-4151-a57e-f9a93312817e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:54.693152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:54.720899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:54.721582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:54.721861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:54.722138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:54.722978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:54.723295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:54.724903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:54.742982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 547/1092] 10.222.0.87 () {56 vars in 1133 bytes} [Fri Jun 17 12:39:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:54.749463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:54.855090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02168fb8-697d-4cb6-8430-088ee19cfaa8 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:54.890234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278cfec-280b-450a-8b19-07ff852af87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:54.890881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278cfec-280b-450a-8b19-07ff852af87b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:54.891160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278cfec-280b-450a-8b19-07ff852af87b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:54.891428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278cfec-280b-450a-8b19-07ff852af87b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:54.956632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:54.960740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278cfec-280b-450a-8b19-07ff852af87b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:54.962509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278cfec-280b-450a-8b19-07ff852af87b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.062073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 546/1093] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:55.068131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.178758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27625fe7-5ea0-4ae0-844e-9940f5c3e9a1 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.210544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e204-1a02-41d4-a964-ff5a68518332 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.210758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e204-1a02-41d4-a964-ff5a68518332 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.211038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e204-1a02-41d4-a964-ff5a68518332 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.212481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 547/1094] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:55.214038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.214852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.215135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.215441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.217594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a1d6-0751-4ab3-9602-3cafb46f1875 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.217874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a1d6-0751-4ab3-9602-3cafb46f1875 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.218149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a1d6-0751-4ab3-9602-3cafb46f1875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.226633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:55.228125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.234923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-5930a1d6-0751-4ab3-9602-3cafb46f1875 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:55.236137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5930a1d6-0751-4ab3-9602-3cafb46f1875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:39:55.245903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:55.246886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:55.248016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:39:55.248016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:39:55.248016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:39:55.248016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:55.248016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 548/1095] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:55.253117 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.271713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:55.306763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 548/1096] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:39:55.330108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930a1d6-0751-4ab3-9602-3cafb46f1875 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.352916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d11f4-c9ba-4d46-af0d-80cebdeb3d34 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.357360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:55.392995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.393670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.393958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.394229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.394958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:55.395265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:55.396652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.407946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 549/1097] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:55.413478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.431972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930a1d6-0751-4ab3-9602-3cafb46f1875 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.465582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.466362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.466647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.466932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.476678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:39:55.484473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:55.485434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:39:55.486256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 549/1098] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:39:55.491583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:55.491706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.514976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99608e-9d74-412e-8994-b347d79672fc None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.547980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.548775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.549054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.549322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.555872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:55.557318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:39:55.571719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:39:55.572686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:39:55.573578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 550/1099] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:39:55.578787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:55.578958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.591545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165a8b9-788f-4662-98c8-7100304a93ac None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.626205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.626877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.627155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.627419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.628196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:55.628484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:55.629862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.641596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 550/1100] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:55.644406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.683781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adacf56-9894-4e37-a539-78b9925d935d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.718558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.719162 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.719452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.719733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.720532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:55.720846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:55.722332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.725237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:55.735476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 551/1101] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:55.741566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.742670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3824464-8e0f-4cd8-8b34-a213545cddb1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.779848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.780730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.781007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.781259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] PATH_INFO: `/v3/users/dfd8e36e9cd4438297e23716269eb6a5` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.782516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] RBAC: Authorizing `identity:get_user(user_id=dfd8e36e9cd4438297e23716269eb6a5)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:55.784005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.785929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 551/1102] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/users/dfd8e36e9cd4438297e23716269eb6a5 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:55.791188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:55.810816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:55.842873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77491a-7497-493f-af0e-d856e8d2ba7c None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.877942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.878609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.878900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.879179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:55.880059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] RBAC: Adding query filter params (role.id=9357b61d1a7c4acbb490a56dd78c1784, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=abaa59b4e88642c681e9563295f6fad9) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:55.880379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:55.881847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:55.888133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 552/1103] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/role_assignments?user.id=abaa59b4e88642c681e9563295f6fad9&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=9357b61d1a7c4acbb490a56dd78c1784 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:39:55.889586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8987ac5-fe9d-4898-9ae5-de51a3d2939f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:55.926972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:55.927674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:55.927961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:55.928239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:56.002716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:56.004271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:56.023859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:56.110218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 552/1104] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:39:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:39:56.116223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:56.177038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:56.221143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812a54a-6cf4-4696-982e-b20e610e3a1f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:56.254594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80edfa-486c-432d-8bc0-63f4c4d984f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:56.255449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80edfa-486c-432d-8bc0-63f4c4d984f5 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:56.255735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80edfa-486c-432d-8bc0-63f4c4d984f5 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:56.256011 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80edfa-486c-432d-8bc0-63f4c4d984f5 None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:56.257211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80edfa-486c-432d-8bc0-63f4c4d984f5 None admin] RBAC: Authorizing `identity:get_project(project_id=5984bda6cb484d85ac14b873bb27fe81)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:56.258654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80edfa-486c-432d-8bc0-63f4c4d984f5 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:56.260419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 553/1105] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:39:56 2022] GET /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:56.266627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:56.368354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd10627-b02a-4980-b5d3-104023ff6721 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:56.386333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:56.404671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae80c70-5cad-4de7-8e0a-515dde0bb0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:56.405338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae80c70-5cad-4de7-8e0a-515dde0bb0bb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:56.405621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae80c70-5cad-4de7-8e0a-515dde0bb0bb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:56.405897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae80c70-5cad-4de7-8e0a-515dde0bb0bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:56.406738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae80c70-5cad-4de7-8e0a-515dde0bb0bb None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=5984bda6cb484d85ac14b873bb27fe81, user.id=dfd8e36e9cd4438297e23716269eb6a5) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:39:56.407047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae80c70-5cad-4de7-8e0a-515dde0bb0bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:56.408578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae80c70-5cad-4de7-8e0a-515dde0bb0bb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:56.414662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 553/1106] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:39:56 2022] GET /identity/v3/role_assignments?user.id=dfd8e36e9cd4438297e23716269eb6a5&scope.project.id=5984bda6cb484d85ac14b873bb27fe81&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:58.570316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269eba-745e-40e5-979c-029945be228e None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:58.571190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269eba-745e-40e5-979c-029945be228e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:58.571190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269eba-745e-40e5-979c-029945be228e None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:58.573273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 554/1107] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:39:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:39:58.578922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987e8e7-ce52-4bce-ae95-c12851c3c9b6 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:58.579389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987e8e7-ce52-4bce-ae95-c12851c3c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:58.579484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987e8e7-ce52-4bce-ae95-c12851c3c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:58.598047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e987e8e7-ce52-4bce-ae95-c12851c3c9b6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:39:58.599241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e987e8e7-ce52-4bce-ae95-c12851c3c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:39:58.668050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 554/1108] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:39:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:39:58.694320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1269eba-745e-40e5-979c-029945be228e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:58.824976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1269eba-745e-40e5-979c-029945be228e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:58.851411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:58.860337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23641e-03a3-403f-815b-bf21df94cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:58.861135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23641e-03a3-403f-815b-bf21df94cec0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:58.861417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23641e-03a3-403f-815b-bf21df94cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:58.861693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23641e-03a3-403f-815b-bf21df94cec0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:58.862936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23641e-03a3-403f-815b-bf21df94cec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:58.864420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23641e-03a3-403f-815b-bf21df94cec0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:58.866077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 555/1109] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:39:58.872372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987e8e7-ce52-4bce-ae95-c12851c3c9b6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:39:58.986496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987e8e7-ce52-4bce-ae95-c12851c3c9b6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:39:58.999536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:39:59.022060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42326d77-bf4c-4600-b809-e218a4af0553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:39:59.022729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42326d77-bf4c-4600-b809-e218a4af0553 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:39:59.023033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42326d77-bf4c-4600-b809-e218a4af0553 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:39:59.023451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42326d77-bf4c-4600-b809-e218a4af0553 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:39:59.024165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42326d77-bf4c-4600-b809-e218a4af0553 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:39:59.025797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42326d77-bf4c-4600-b809-e218a4af0553 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:39:59.051030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 555/1110] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:39:58 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:40:01.238552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bc3f-3b9b-41ee-82dc-c71efdc01bdc None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:01.239505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bc3f-3b9b-41ee-82dc-c71efdc01bdc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:01.239505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bc3f-3b9b-41ee-82dc-c71efdc01bdc None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:01.241756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 556/1111] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:01.248229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcbc4-51a6-4a82-b110-02709853a94b None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:01.248671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcbc4-51a6-4a82-b110-02709853a94b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:01.248794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcbc4-51a6-4a82-b110-02709853a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:01.276887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-5c7fcbc4-51a6-4a82-b110-02709853a94b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:01.278718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5c7fcbc4-51a6-4a82-b110-02709853a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:01.356034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 556/1112] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:01.379732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621bc3f-3b9b-41ee-82dc-c71efdc01bdc None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:01.484095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621bc3f-3b9b-41ee-82dc-c71efdc01bdc None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:01.519683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:01.520561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:01.520841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:01.521115 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] PATH_INFO: `/v3/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:01.522647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] RBAC: Authorizing `identity:get_role(role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:01.524065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:01.525453 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 557/1113] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:01 2022] GET /identity/v3/roles/20865fed4c894cccb7f1c223e95353ec => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:01.532372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fcbc4-51a6-4a82-b110-02709853a94b None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:01.634441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:01.638226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fcbc4-51a6-4a82-b110-02709853a94b None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:01.676921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:01.677800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:01.678088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:01.678364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] PATH_INFO: `/v3/users/b5935ffee23e4326a235d6d3f725da7d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:01.693269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] RBAC: Authorizing `identity:get_user(user_id=b5935ffee23e4326a235d6d3f725da7d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:01.694728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:01.696410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 557/1114] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:01 2022] GET /identity/v3/users/b5935ffee23e4326a235d6d3f725da7d => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:01.702732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:01.768200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:01.810972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b2424-b5cb-401d-b33e-152ce01a578f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:01.851705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:01.852432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:01.852721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:01.853130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:01.932442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:01.933892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:02.004371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:02.043736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 558/1115] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:40:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:40:02.051127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:02.159755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5ccec-d9ea-4b21-8333-794d5e4e2dbb None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:02.211974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18b50a-1703-40c3-b4ef-86c832b95c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:02.212768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18b50a-1703-40c3-b4ef-86c832b95c7c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:02.213052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18b50a-1703-40c3-b4ef-86c832b95c7c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:02.213329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18b50a-1703-40c3-b4ef-86c832b95c7c None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:02.214644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18b50a-1703-40c3-b4ef-86c832b95c7c None admin] RBAC: Authorizing `identity:get_project(project_id=5984bda6cb484d85ac14b873bb27fe81)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:02.216123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18b50a-1703-40c3-b4ef-86c832b95c7c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:02.218024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 558/1116] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:02 2022] GET /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:02.224427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:02.340580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97961f07-f54a-4901-a8b7-33ead2df5661 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:02.381406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b43a-9a08-48cf-82f0-e3e2ac57a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:02.382055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b43a-9a08-48cf-82f0-e3e2ac57a6bf None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:02.382332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b43a-9a08-48cf-82f0-e3e2ac57a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:02.382602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b43a-9a08-48cf-82f0-e3e2ac57a6bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:02.383459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b43a-9a08-48cf-82f0-e3e2ac57a6bf None admin] RBAC: Adding query filter params (role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=5984bda6cb484d85ac14b873bb27fe81, user.id=b5935ffee23e4326a235d6d3f725da7d) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:02.383770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b43a-9a08-48cf-82f0-e3e2ac57a6bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:02.385192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b43a-9a08-48cf-82f0-e3e2ac57a6bf None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:02.391516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 559/1117] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:40:02 2022] GET /identity/v3/role_assignments?user.id=b5935ffee23e4326a235d6d3f725da7d&scope.project.id=5984bda6cb484d85ac14b873bb27fe81&role.id=20865fed4c894cccb7f1c223e95353ec => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:04.479001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad80ce-adc1-4f68-a030-e9a2682c2d8a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:04.479651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad80ce-adc1-4f68-a030-e9a2682c2d8a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:04.479651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad80ce-adc1-4f68-a030-e9a2682c2d8a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:04.481565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 559/1118] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:04.488685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d108f4-8b83-47ac-a447-cc515ab22350 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:04.489426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d108f4-8b83-47ac-a447-cc515ab22350 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:04.489576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d108f4-8b83-47ac-a447-cc515ab22350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:04.515972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-85d108f4-8b83-47ac-a447-cc515ab22350 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:04.517181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-85d108f4-8b83-47ac-a447-cc515ab22350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:04.541349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:04.588985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 560/1119] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:04.618952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad80ce-adc1-4f68-a030-e9a2682c2d8a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:04.647015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:04.731136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad80ce-adc1-4f68-a030-e9a2682c2d8a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:04.766828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:04.767722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:04.767960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:04.768272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] PATH_INFO: `/v3/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:04.769879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] RBAC: Authorizing `identity:get_role(role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:04.771281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:04.772731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 560/1120] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:04 2022] GET /identity/v3/roles/20865fed4c894cccb7f1c223e95353ec => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:04.779227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d108f4-8b83-47ac-a447-cc515ab22350 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:04.885073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d108f4-8b83-47ac-a447-cc515ab22350 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:04.923073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:04.924139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:04.924415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:04.924686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] PATH_INFO: `/v3/users/b5935ffee23e4326a235d6d3f725da7d` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:04.925919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] RBAC: Authorizing `identity:get_user(user_id=b5935ffee23e4326a235d6d3f725da7d)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:04.927353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:04.929157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 561/1121] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:04 2022] GET /identity/v3/users/b5935ffee23e4326a235d6d3f725da7d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:04.935355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:05.034051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:05.044496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f771b6b-b1d9-4743-92cd-52c5bf624712 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:05.079930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810d34d-aced-45f0-802b-3d84e43efe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:05.080800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810d34d-aced-45f0-802b-3d84e43efe4b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:05.081081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810d34d-aced-45f0-802b-3d84e43efe4b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:05.081356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810d34d-aced-45f0-802b-3d84e43efe4b None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:05.082590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810d34d-aced-45f0-802b-3d84e43efe4b None admin] RBAC: Authorizing `identity:get_project(project_id=5984bda6cb484d85ac14b873bb27fe81)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:05.084172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810d34d-aced-45f0-802b-3d84e43efe4b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:05.085950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 561/1122] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:04 2022] GET /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:05.092498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:05.122801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:05.198816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb401-ffb5-498d-a82a-5dd1b9251740 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:05.237163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2c4a8-9dcf-494c-a360-6e171cda5437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:05.238092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2c4a8-9dcf-494c-a360-6e171cda5437 None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:05.238374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2c4a8-9dcf-494c-a360-6e171cda5437 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:05.238629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2c4a8-9dcf-494c-a360-6e171cda5437 None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81/users/b5935ffee23e4326a235d6d3f725da7d/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:05.241226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2c4a8-9dcf-494c-a360-6e171cda5437 None admin] RBAC: Authorizing `identity:create_grant(project_id=5984bda6cb484d85ac14b873bb27fe81, user_id=b5935ffee23e4326a235d6d3f725da7d, role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:05.242672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2c4a8-9dcf-494c-a360-6e171cda5437 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:05.258743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 562/1123] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:40:05 2022] PUT /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81/users/b5935ffee23e4326a235d6d3f725da7d/roles/20865fed4c894cccb7f1c223e95353ec => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:40:07.350212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb2ec9-9a82-4638-9eec-fefec242c296 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:07.350784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb2ec9-9a82-4638-9eec-fefec242c296 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:07.350784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb2ec9-9a82-4638-9eec-fefec242c296 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:07.352943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 562/1124] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:07.361872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1c82b-c32d-4da3-bc94-ef3590bb0f0f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:07.362284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1c82b-c32d-4da3-bc94-ef3590bb0f0f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:07.362494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1c82b-c32d-4da3-bc94-ef3590bb0f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:07.390928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-12d1c82b-c32d-4da3-bc94-ef3590bb0f0f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:07.392734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-12d1c82b-c32d-4da3-bc94-ef3590bb0f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:07.470396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 563/1125] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:07.499885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb2ec9-9a82-4638-9eec-fefec242c296 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:07.609930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb2ec9-9a82-4638-9eec-fefec242c296 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:07.650966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:07.652186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:07.652552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:07.652919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] PATH_INFO: `/v3/roles/20865fed4c894cccb7f1c223e95353ec` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:07.655413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] RBAC: Authorizing `identity:get_role(role_id=20865fed4c894cccb7f1c223e95353ec)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:07.657588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:07.659717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 563/1126] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:07 2022] GET /identity/v3/roles/20865fed4c894cccb7f1c223e95353ec => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:07.666316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1c82b-c32d-4da3-bc94-ef3590bb0f0f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:07.687079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:07.770787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1c82b-c32d-4da3-bc94-ef3590bb0f0f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:07.808652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:07.809539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:07.809828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:07.810105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] PATH_INFO: `/v3/users/b5935ffee23e4326a235d6d3f725da7d` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:07.811417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] RBAC: Authorizing `identity:get_user(user_id=b5935ffee23e4326a235d6d3f725da7d)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:07.812875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:07.814528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 564/1127] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:07 2022] GET /identity/v3/users/b5935ffee23e4326a235d6d3f725da7d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:07.822954 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:07.853893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:07.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0af3e-f9c8-4f0b-95b9-e86415185742 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:07.962436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:07.963068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:07.963346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:07.963653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:08.032725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:08.034164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:08.065473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:08.136898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 564/1128] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:40:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:40:08.145729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:08.238143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:08.254886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808aef8-f148-405e-812f-0ac9361cdc88 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:08.290022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0baef53-e543-4199-a032-6869cc7f0b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:08.290881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baef53-e543-4199-a032-6869cc7f0b49 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:08.291153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baef53-e543-4199-a032-6869cc7f0b49 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:08.291434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baef53-e543-4199-a032-6869cc7f0b49 None admin] PATH_INFO: `/v3/projects/5984bda6cb484d85ac14b873bb27fe81` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:08.292688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0baef53-e543-4199-a032-6869cc7f0b49 None admin] RBAC: Authorizing `identity:get_project(project_id=5984bda6cb484d85ac14b873bb27fe81)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:08.294114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0baef53-e543-4199-a032-6869cc7f0b49 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:08.296755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 565/1129] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:08 2022] GET /identity/v3/projects/5984bda6cb484d85ac14b873bb27fe81 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:08.303877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:08.413151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1d6e-5fdf-48f2-9c8a-e20952173d7a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:08.440432 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:08.449076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268b89-976a-431d-ba6f-080de450f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:08.449728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268b89-976a-431d-ba6f-080de450f4ab None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:08.450011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268b89-976a-431d-ba6f-080de450f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:08.450287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268b89-976a-431d-ba6f-080de450f4ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:08.451117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268b89-976a-431d-ba6f-080de450f4ab None admin] RBAC: Adding query filter params (role.id=20865fed4c894cccb7f1c223e95353ec, scope.project.id=5984bda6cb484d85ac14b873bb27fe81, user.id=b5935ffee23e4326a235d6d3f725da7d) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:08.451618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268b89-976a-431d-ba6f-080de450f4ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:08.453056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268b89-976a-431d-ba6f-080de450f4ab None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:08.459019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 565/1130] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:40:08 2022] GET /identity/v3/role_assignments?user.id=b5935ffee23e4326a235d6d3f725da7d&scope.project.id=5984bda6cb484d85ac14b873bb27fe81&role.id=20865fed4c894cccb7f1c223e95353ec => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:10.579710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f79a5-b471-459a-bfe5-12c16960af11 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:10.580274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f79a5-b471-459a-bfe5-12c16960af11 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:10.580274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f79a5-b471-459a-bfe5-12c16960af11 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:10.581667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 566/1131] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:10.588047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2f77a-d5b6-4101-97b3-483da0642621 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:10.588747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2f77a-d5b6-4101-97b3-483da0642621 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:10.588941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2f77a-d5b6-4101-97b3-483da0642621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:10.612938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-85e2f77a-d5b6-4101-97b3-483da0642621 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:10.614142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-85e2f77a-d5b6-4101-97b3-483da0642621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:10.687051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 566/1132] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:10.716250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f79a5-b471-459a-bfe5-12c16960af11 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:10.836597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f79a5-b471-459a-bfe5-12c16960af11 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:10.872203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e8fd-a26c-4d3c-a2a0-b31b6a2cd604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:10.872988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e8fd-a26c-4d3c-a2a0-b31b6a2cd604 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:10.873336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e8fd-a26c-4d3c-a2a0-b31b6a2cd604 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:10.873755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e8fd-a26c-4d3c-a2a0-b31b6a2cd604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:10.874890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30e8fd-a26c-4d3c-a2a0-b31b6a2cd604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:10.876354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30e8fd-a26c-4d3c-a2a0-b31b6a2cd604 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:10.877987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 567/1133] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:40:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:10.887815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2f77a-d5b6-4101-97b3-483da0642621 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:10.996091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2f77a-d5b6-4101-97b3-483da0642621 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:11.019117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:11.034193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b4d17-7d25-4f87-b416-930a9325fd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:11.034885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4d17-7d25-4f87-b416-930a9325fd50 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:11.035171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4d17-7d25-4f87-b416-930a9325fd50 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:11.035472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4d17-7d25-4f87-b416-930a9325fd50 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:11.036190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b4d17-7d25-4f87-b416-930a9325fd50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:11.037634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b4d17-7d25-4f87-b416-930a9325fd50 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:11.074183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 567/1134] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:40:10 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:40:13.243057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6920-6fc5-4558-a100-8c667c312574 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:13.243679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6920-6fc5-4558-a100-8c667c312574 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:13.243679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6920-6fc5-4558-a100-8c667c312574 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:13.245072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 568/1135] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:13.251739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0d1c-e299-433f-92d4-ef53c7331d34 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:13.252458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0d1c-e299-433f-92d4-ef53c7331d34 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:13.252596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0d1c-e299-433f-92d4-ef53c7331d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:13.281156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-143f0d1c-e299-433f-92d4-ef53c7331d34 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:13.282372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-143f0d1c-e299-433f-92d4-ef53c7331d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:13.353377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 568/1136] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:13.379250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca6920-6fc5-4558-a100-8c667c312574 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:13.401797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:13.485386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca6920-6fc5-4558-a100-8c667c312574 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:13.522544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205a961-8901-4115-8fde-f5d28743a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:13.523345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205a961-8901-4115-8fde-f5d28743a50d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:13.523667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205a961-8901-4115-8fde-f5d28743a50d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:13.523942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205a961-8901-4115-8fde-f5d28743a50d None admin] PATH_INFO: `/v3/domains/default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:13.525273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205a961-8901-4115-8fde-f5d28743a50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:13.526711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205a961-8901-4115-8fde-f5d28743a50d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:13.528374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 569/1137] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:40:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:13.534691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0d1c-e299-433f-92d4-ef53c7331d34 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:13.645094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0d1c-e299-433f-92d4-ef53c7331d34 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:13.647614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:13.681646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66190995-a095-496d-bb89-7b732dc8f658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:13.682350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66190995-a095-496d-bb89-7b732dc8f658 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:13.682629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66190995-a095-496d-bb89-7b732dc8f658 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:13.682903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66190995-a095-496d-bb89-7b732dc8f658 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:13.683644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66190995-a095-496d-bb89-7b732dc8f658 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:13.685079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66190995-a095-496d-bb89-7b732dc8f658 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:13.708490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 569/1138] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:40:13 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:40:15.848607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9b5bf-1e17-48e3-b55e-0ac94dcde186 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:15.849147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9b5bf-1e17-48e3-b55e-0ac94dcde186 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:15.849147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9b5bf-1e17-48e3-b55e-0ac94dcde186 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:15.851208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 570/1139] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:15.859779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc0388-c073-4e2e-90c3-6a5b39253a32 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:15.860183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc0388-c073-4e2e-90c3-6a5b39253a32 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:15.860424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc0388-c073-4e2e-90c3-6a5b39253a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:15.887493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-44bc0388-c073-4e2e-90c3-6a5b39253a32 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:15.889417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-44bc0388-c073-4e2e-90c3-6a5b39253a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:15.965899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 570/1140] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:15.990496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9b5bf-1e17-48e3-b55e-0ac94dcde186 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:16.107126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:16.123284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9b5bf-1e17-48e3-b55e-0ac94dcde186 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:16.160985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:16.162139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:16.162560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:16.162933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:16.181238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:40:16.191342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:40:16.192459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:40:16.193669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 571/1141] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:40:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:40:16.199295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc0388-c073-4e2e-90c3-6a5b39253a32 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:16.311745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:16.324146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc0388-c073-4e2e-90c3-6a5b39253a32 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:16.365131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:16.365765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:16.366056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:16.366330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:16.367087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:16.367413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:16.368829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:16.378490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 571/1142] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:16.384969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:16.511340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1575b-2c15-47ee-8ac0-1c06096f84ac None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:16.546124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:16.547166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:16.547476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:16.547748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] PATH_INFO: `/v3/users/b11196da9ca047ba9a2adaa8b9aa6f60` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:16.562517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] RBAC: Authorizing `identity:get_user(user_id=b11196da9ca047ba9a2adaa8b9aa6f60)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:16.564001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:16.565660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 572/1143] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:16 2022] GET /identity/v3/users/b11196da9ca047ba9a2adaa8b9aa6f60 => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:16.572335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:16.694458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9049cb-3b9d-4ad8-afb1-4cc02e19108e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:16.745430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:16.746092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:16.746380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:16.746660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:16.769016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:16.818980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:16.820507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:16.926980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 572/1144] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:40:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:40:16.933368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:16.942595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:17.037001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c979-07e1-496e-958f-c4a5e8d309d9 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:17.074887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cea88-5e51-4035-9cd7-d57aed11629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:17.075794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cea88-5e51-4035-9cd7-d57aed11629b None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:17.076079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cea88-5e51-4035-9cd7-d57aed11629b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:17.076351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cea88-5e51-4035-9cd7-d57aed11629b None admin] PATH_INFO: `/v3/projects/230013f327764f558981ce5788383521` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:17.077820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cea88-5e51-4035-9cd7-d57aed11629b None admin] RBAC: Authorizing `identity:get_project(project_id=230013f327764f558981ce5788383521)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:17.079261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cea88-5e51-4035-9cd7-d57aed11629b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:17.081036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 573/1145] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:16 2022] GET /identity/v3/projects/230013f327764f558981ce5788383521 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:17.088161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:17.147109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:17.199271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361335de-9051-4e57-801d-741b16ee0fb7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:17.235052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90e192-97d8-44f9-82e6-3df15f780a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:17.235738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e192-97d8-44f9-82e6-3df15f780a10 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:17.236021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e192-97d8-44f9-82e6-3df15f780a10 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:17.236298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e192-97d8-44f9-82e6-3df15f780a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:17.237136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e192-97d8-44f9-82e6-3df15f780a10 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=230013f327764f558981ce5788383521, user.id=b11196da9ca047ba9a2adaa8b9aa6f60) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:17.237453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e192-97d8-44f9-82e6-3df15f780a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:17.238893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e192-97d8-44f9-82e6-3df15f780a10 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:17.244587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 573/1146] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:40:17 2022] GET /identity/v3/role_assignments?user.id=b11196da9ca047ba9a2adaa8b9aa6f60&scope.project.id=230013f327764f558981ce5788383521&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:19.337223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c119-6cbb-4908-866b-6f4db7632792 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:19.338100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c119-6cbb-4908-866b-6f4db7632792 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:19.338100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c119-6cbb-4908-866b-6f4db7632792 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:19.340275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 574/1147] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:19.346990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d0ec8-00d0-4ffc-809f-8a3a102efa09 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:19.347470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d0ec8-00d0-4ffc-809f-8a3a102efa09 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:19.347580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d0ec8-00d0-4ffc-809f-8a3a102efa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:19.365130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-272d0ec8-00d0-4ffc-809f-8a3a102efa09 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:19.366244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-272d0ec8-00d0-4ffc-809f-8a3a102efa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:19.434883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 574/1148] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 12:40:19.458151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581c119-6cbb-4908-866b-6f4db7632792 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:19.538871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:19.567388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581c119-6cbb-4908-866b-6f4db7632792 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:19.603699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:19.604483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:19.604761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:19.605030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:19.616390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:40:19.623966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:40:19.625057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:19.626041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:40:19.627553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 575/1149] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:40:19.631869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d0ec8-00d0-4ffc-809f-8a3a102efa09 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:19.681577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:19.749136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d0ec8-00d0-4ffc-809f-8a3a102efa09 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:19.784618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:19.785295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:19.785580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:19.785853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:19.786608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:19.786915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:19.788474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:19.797067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 575/1150] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:40:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:19.803997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:19.928035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019244df-7f53-4b25-9daa-b3915f0de92d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:19.964552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:19.965413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:19.965690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:19.965963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] PATH_INFO: `/v3/users/b11196da9ca047ba9a2adaa8b9aa6f60` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:19.967251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] RBAC: Authorizing `identity:get_user(user_id=b11196da9ca047ba9a2adaa8b9aa6f60)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:19.968860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:19.970489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 576/1151] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:19 2022] GET /identity/v3/users/b11196da9ca047ba9a2adaa8b9aa6f60 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:19.976651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:20.083524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76cd2c-4459-4106-9cd1-1e0079ac068f None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:20.093077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:20.126927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24998e-e138-4408-a587-84e331480f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:20.127851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24998e-e138-4408-a587-84e331480f56 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:20.128139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24998e-e138-4408-a587-84e331480f56 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:20.128419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24998e-e138-4408-a587-84e331480f56 None admin] PATH_INFO: `/v3/projects/230013f327764f558981ce5788383521` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:20.129642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24998e-e138-4408-a587-84e331480f56 None admin] RBAC: Authorizing `identity:get_project(project_id=230013f327764f558981ce5788383521)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:20.131092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24998e-e138-4408-a587-84e331480f56 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:20.132860 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 576/1152] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:19 2022] GET /identity/v3/projects/230013f327764f558981ce5788383521 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:20.139155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:20.142594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:20.245645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e5952-6c73-4157-b477-bc89f19af14f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:20.280527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f1628-5633-47d2-a770-a01a0ae4f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:20.281504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f1628-5633-47d2-a770-a01a0ae4f03e None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:20.281798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f1628-5633-47d2-a770-a01a0ae4f03e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:20.282068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f1628-5633-47d2-a770-a01a0ae4f03e None admin] PATH_INFO: `/v3/projects/230013f327764f558981ce5788383521/users/b11196da9ca047ba9a2adaa8b9aa6f60/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:20.284164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f1628-5633-47d2-a770-a01a0ae4f03e None admin] RBAC: Authorizing `identity:create_grant(project_id=230013f327764f558981ce5788383521, user_id=b11196da9ca047ba9a2adaa8b9aa6f60, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:20.285613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f1628-5633-47d2-a770-a01a0ae4f03e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:20.302700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 577/1153] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:40:20 2022] PUT /identity/v3/projects/230013f327764f558981ce5788383521/users/b11196da9ca047ba9a2adaa8b9aa6f60/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:40:22.403796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164f57-6246-4d3d-a8ce-1b9e654c25d6 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:22.404339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164f57-6246-4d3d-a8ce-1b9e654c25d6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:22.404339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164f57-6246-4d3d-a8ce-1b9e654c25d6 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:22.405644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 577/1154] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:22.412121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec87de-a039-4ef7-8ee7-4887a15849d2 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:22.412799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec87de-a039-4ef7-8ee7-4887a15849d2 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:22.413052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec87de-a039-4ef7-8ee7-4887a15849d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:22.435740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-97ec87de-a039-4ef7-8ee7-4887a15849d2 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:22.436938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-97ec87de-a039-4ef7-8ee7-4887a15849d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:22.499978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:22.512148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 578/1155] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:22.535621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164f57-6246-4d3d-a8ce-1b9e654c25d6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:22.649055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164f57-6246-4d3d-a8ce-1b9e654c25d6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:22.691102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:22.692259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:22.692643 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:22.693039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:22.708207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:40:22.715795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:40:22.716863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:40:22.717891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 578/1156] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:40:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:40:22.725505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec87de-a039-4ef7-8ee7-4887a15849d2 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:22.834173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec87de-a039-4ef7-8ee7-4887a15849d2 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:22.871089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:22.871793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:22.872074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:22.872352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:22.873109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:22.873429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:22.874867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:22.883772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 579/1157] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:40:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:22.893934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:22.934536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:23.002234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431506c-4fd2-4044-b2bf-adac9479a24a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:23.049197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:23.050069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:23.050355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:23.050633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] PATH_INFO: `/v3/users/b11196da9ca047ba9a2adaa8b9aa6f60` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:23.051963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=b11196da9ca047ba9a2adaa8b9aa6f60)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:23.053416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:23.055032 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 579/1158] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:22 2022] GET /identity/v3/users/b11196da9ca047ba9a2adaa8b9aa6f60 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:23.063754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:23.128672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:23.180056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afd49e-a491-4dd2-a720-3ba7fb466953 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:23.215018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:23.215695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:23.215980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:23.216253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:23.286292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:23.287697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:23.344047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:23.393350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 580/1159] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:40:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 12:40:23.399962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:23.501456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed586-941a-46fb-be05-bf9e2a161cf1 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:23.503862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:23.536760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ffbf8-d747-47f5-971d-f449032f4293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:23.537551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ffbf8-d747-47f5-971d-f449032f4293 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:23.537833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ffbf8-d747-47f5-971d-f449032f4293 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:23.538106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ffbf8-d747-47f5-971d-f449032f4293 None admin] PATH_INFO: `/v3/projects/230013f327764f558981ce5788383521` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:23.539337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ffbf8-d747-47f5-971d-f449032f4293 None admin] RBAC: Authorizing `identity:get_project(project_id=230013f327764f558981ce5788383521)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:23.540820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ffbf8-d747-47f5-971d-f449032f4293 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:23.542492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 580/1160] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:23 2022] GET /identity/v3/projects/230013f327764f558981ce5788383521 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:23.549594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:23.651887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc845e-acb9-4aad-9ebd-1b5acebefb19 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:23.688202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86724-2ca7-46f0-829e-ae52af88fc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:23.688847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86724-2ca7-46f0-829e-ae52af88fc8f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:23.689127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86724-2ca7-46f0-829e-ae52af88fc8f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:23.689400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86724-2ca7-46f0-829e-ae52af88fc8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:23.690234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef86724-2ca7-46f0-829e-ae52af88fc8f None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=230013f327764f558981ce5788383521, user.id=b11196da9ca047ba9a2adaa8b9aa6f60) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:23.690540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef86724-2ca7-46f0-829e-ae52af88fc8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:23.692019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef86724-2ca7-46f0-829e-ae52af88fc8f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:23.698410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 581/1161] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:40:23 2022] GET /identity/v3/role_assignments?user.id=b11196da9ca047ba9a2adaa8b9aa6f60&scope.project.id=230013f327764f558981ce5788383521&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:25.842575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c2c3-be82-4da5-a0da-a34d5d585a8c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:25.843808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c2c3-be82-4da5-a0da-a34d5d585a8c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:25.843808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c2c3-be82-4da5-a0da-a34d5d585a8c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:25.845058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 581/1162] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:25.851198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda706c-009f-489b-b842-a77fb6440d23 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:25.851689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda706c-009f-489b-b842-a77fb6440d23 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:25.851791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda706c-009f-489b-b842-a77fb6440d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:25.879063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8fda706c-009f-489b-b842-a77fb6440d23 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:25.880270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8fda706c-009f-489b-b842-a77fb6440d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:25.894135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:25.951798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 582/1163] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:25.981172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c2c3-be82-4da5-a0da-a34d5d585a8c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:26.090106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c2c3-be82-4da5-a0da-a34d5d585a8c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:26.128103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:26.128758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:26.129031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:26.129303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:26.173457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:26.201684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:26.203315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:26.306871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 582/1164] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:40:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 12:40:26.315884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda706c-009f-489b-b842-a77fb6440d23 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:26.418232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda706c-009f-489b-b842-a77fb6440d23 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:26.453121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f75453-cd5b-453e-9acc-444c3dac168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:26.453917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f75453-cd5b-453e-9acc-444c3dac168e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:26.454199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f75453-cd5b-453e-9acc-444c3dac168e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:26.454472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f75453-cd5b-453e-9acc-444c3dac168e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:26.461378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f75453-cd5b-453e-9acc-444c3dac168e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:26.462827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f75453-cd5b-453e-9acc-444c3dac168e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-15f75453-cd5b-453e-9acc-444c3dac168e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:40:26.477072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:40:26.478279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:40:26.479682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 583/1165] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:40:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:40:26.486830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:26.556302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:26.587102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e04dd7-9c34-4a8d-a313-125925aae0b0 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:26.622160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfdba1-d743-4acd-98c0-c60a94fe9748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:26.622814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfdba1-d743-4acd-98c0-c60a94fe9748 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:26.623095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfdba1-d743-4acd-98c0-c60a94fe9748 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:26.623385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfdba1-d743-4acd-98c0-c60a94fe9748 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:26.624126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfdba1-d743-4acd-98c0-c60a94fe9748 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:26.624437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfdba1-d743-4acd-98c0-c60a94fe9748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:26.626106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfdba1-d743-4acd-98c0-c60a94fe9748 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:26.633233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 583/1166] 10.222.0.87 () {56 vars in 1115 bytes} [Fri Jun 17 12:40:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:28.749339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7f1e-18be-444c-85f3-78023954c154 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:28.750251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7f1e-18be-444c-85f3-78023954c154 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:28.751076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7f1e-18be-444c-85f3-78023954c154 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:28.753353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 584/1167] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:28.762250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd90458-de89-4384-a577-c2621ecb7dd3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:28.762937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd90458-de89-4384-a577-c2621ecb7dd3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:28.763121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd90458-de89-4384-a577-c2621ecb7dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:28.783423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-efd90458-de89-4384-a577-c2621ecb7dd3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:28.784642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-efd90458-de89-4384-a577-c2621ecb7dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:28.862659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 584/1168] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:28.895569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a7f1e-18be-444c-85f3-78023954c154 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:28.912018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:29.017780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a7f1e-18be-444c-85f3-78023954c154 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:29.055389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d2b94-22f2-4665-aa75-3bbac96de40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:29.056066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d2b94-22f2-4665-aa75-3bbac96de40b None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:29.056343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d2b94-22f2-4665-aa75-3bbac96de40b None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:29.056616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d2b94-22f2-4665-aa75-3bbac96de40b None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:29.057391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d2b94-22f2-4665-aa75-3bbac96de40b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:29.058759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d2b94-22f2-4665-aa75-3bbac96de40b None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:29.075545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 585/1169] 10.222.0.87 () {60 vars in 1138 bytes} [Fri Jun 17 12:40:28 2022] POST /identity/v3/domains => generated 267 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:40:31.232119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0637821-cea7-4ccf-9dde-b8d578a5ee82 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:31.232718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0637821-cea7-4ccf-9dde-b8d578a5ee82 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:31.232718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0637821-cea7-4ccf-9dde-b8d578a5ee82 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:31.234029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 585/1170] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:31.241193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2536-fb8c-46a3-9694-566600d48854 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:31.241903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2536-fb8c-46a3-9694-566600d48854 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:31.242084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2536-fb8c-46a3-9694-566600d48854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:31.266570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8bfa2536-fb8c-46a3-9694-566600d48854 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:31.267807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8bfa2536-fb8c-46a3-9694-566600d48854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:31.334092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:31.339673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 586/1171] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:31.364039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0637821-cea7-4ccf-9dde-b8d578a5ee82 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:31.455364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:31.495323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0637821-cea7-4ccf-9dde-b8d578a5ee82 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:31.530247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175fc1f-808e-42fc-8736-91f561eb30ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:31.531101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fc1f-808e-42fc-8736-91f561eb30ca None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:31.531413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fc1f-808e-42fc-8736-91f561eb30ca None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:31.531678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fc1f-808e-42fc-8736-91f561eb30ca None admin] PATH_INFO: `/v3/domains/b4525f33860b49baae4276870991dab7` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:31.542609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175fc1f-808e-42fc-8736-91f561eb30ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=b4525f33860b49baae4276870991dab7)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:31.544095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175fc1f-808e-42fc-8736-91f561eb30ca None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:31.545715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 586/1172] 10.222.0.87 () {56 vars in 1183 bytes} [Fri Jun 17 12:40:31 2022] GET /identity/v3/domains/b4525f33860b49baae4276870991dab7 => generated 267 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:31.552908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa2536-fb8c-46a3-9694-566600d48854 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:31.666244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa2536-fb8c-46a3-9694-566600d48854 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:31.703065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7b098-a7a0-4fb4-9a96-a7f9b7d6b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:31.703761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7b098-a7a0-4fb4-9a96-a7f9b7d6b32a None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:31.704045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7b098-a7a0-4fb4-9a96-a7f9b7d6b32a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:31.704323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7b098-a7a0-4fb4-9a96-a7f9b7d6b32a None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:31.705033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7b098-a7a0-4fb4-9a96-a7f9b7d6b32a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:31.706713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7b098-a7a0-4fb4-9a96-a7f9b7d6b32a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:31.736859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 587/1173] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:40:31 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:40:33.871364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d1ef6-c76b-4534-9598-191bfbaf3274 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:33.872306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d1ef6-c76b-4534-9598-191bfbaf3274 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:33.872306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d1ef6-c76b-4534-9598-191bfbaf3274 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:33.874568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 587/1174] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:33.880165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7f1ba-7e27-4a68-80c1-f42f88cd7fbd None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:33.880602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7f1ba-7e27-4a68-80c1-f42f88cd7fbd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:33.880707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7f1ba-7e27-4a68-80c1-f42f88cd7fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:33.899224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-31f7f1ba-7e27-4a68-80c1-f42f88cd7fbd None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:33.900445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-31f7f1ba-7e27-4a68-80c1-f42f88cd7fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:33.947866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:33.970634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 588/1175] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:33.996411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d1ef6-c76b-4534-9598-191bfbaf3274 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:34.110354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d1ef6-c76b-4534-9598-191bfbaf3274 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:34.141298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:34.155029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7c455-c1ce-4e1a-8e32-c4ca6d5cb0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:34.155948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7c455-c1ce-4e1a-8e32-c4ca6d5cb0c6 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:34.156238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7c455-c1ce-4e1a-8e32-c4ca6d5cb0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:34.156539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7c455-c1ce-4e1a-8e32-c4ca6d5cb0c6 None admin] PATH_INFO: `/v3/domains/b4525f33860b49baae4276870991dab7` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:34.157965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7c455-c1ce-4e1a-8e32-c4ca6d5cb0c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b4525f33860b49baae4276870991dab7)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:34.159578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7c455-c1ce-4e1a-8e32-c4ca6d5cb0c6 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:34.161259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 588/1176] 10.222.0.87 () {56 vars in 1183 bytes} [Fri Jun 17 12:40:33 2022] GET /identity/v3/domains/b4525f33860b49baae4276870991dab7 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:34.167291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7f1ba-7e27-4a68-80c1-f42f88cd7fbd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:34.268850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7f1ba-7e27-4a68-80c1-f42f88cd7fbd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:34.304682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d9c03-3799-4dde-9ade-ca4f0f0a0f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:34.305346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9c03-3799-4dde-9ade-ca4f0f0a0f2d None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:34.305635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9c03-3799-4dde-9ade-ca4f0f0a0f2d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:34.305916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9c03-3799-4dde-9ade-ca4f0f0a0f2d None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:34.306630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d9c03-3799-4dde-9ade-ca4f0f0a0f2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:34.308101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d9c03-3799-4dde-9ade-ca4f0f0a0f2d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:34.330641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 589/1177] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:40:34 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:40:36.473137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bd47c-68e6-4dca-bfae-e229b8af33e8 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:36.473665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bd47c-68e6-4dca-bfae-e229b8af33e8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:36.473665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bd47c-68e6-4dca-bfae-e229b8af33e8 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:36.475011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 589/1178] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:36.481491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef847e-b13a-46aa-b2db-2c5857585e36 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:36.482185 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef847e-b13a-46aa-b2db-2c5857585e36 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:36.482428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef847e-b13a-46aa-b2db-2c5857585e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:36.508146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9fef847e-b13a-46aa-b2db-2c5857585e36 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:36.509349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9fef847e-b13a-46aa-b2db-2c5857585e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:36.539637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:36.581345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 590/1179] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:36.612035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bd47c-68e6-4dca-bfae-e229b8af33e8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:36.716498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bd47c-68e6-4dca-bfae-e229b8af33e8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:36.751406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:36.752194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:36.752475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:36.752755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:36.764190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:40:36.771863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:40:36.772985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:40:36.773870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 590/1180] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:40:36.779607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef847e-b13a-46aa-b2db-2c5857585e36 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:36.881102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef847e-b13a-46aa-b2db-2c5857585e36 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:36.916857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:36.917519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:36.917799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:36.918079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:36.918833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:36.919146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:36.920550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:36.929183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 591/1181] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:36.935469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:36.991310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:37.034510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93576ab4-bb4d-44c4-bc3d-d9ea4fa1aa28 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:37.069626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:37.070494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:37.070774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:37.071052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] PATH_INFO: `/v3/users/77bda763d13048749354bf5c1ae1d59a` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:37.084545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] RBAC: Authorizing `identity:get_user(user_id=77bda763d13048749354bf5c1ae1d59a)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:37.085994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:37.087658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 591/1182] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:36 2022] GET /identity/v3/users/77bda763d13048749354bf5c1ae1d59a => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:37.093756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:37.125068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:37.202171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0959d7-39f1-446c-bfc9-07e71999aee1 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:37.241350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:37.242010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:37.242292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:37.242568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:37.325641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:37.327018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:37.353675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:37.427421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 592/1183] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:40:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:40:37.433875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:37.538218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81029e-9a63-4cb9-b4b0-f3dd8052d99c None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:37.552625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:37.574993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73717b-3a3f-4891-9452-297a7cb71138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:37.575945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73717b-3a3f-4891-9452-297a7cb71138 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:37.576166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73717b-3a3f-4891-9452-297a7cb71138 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:37.576436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73717b-3a3f-4891-9452-297a7cb71138 None admin] PATH_INFO: `/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:37.577743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73717b-3a3f-4891-9452-297a7cb71138 None admin] RBAC: Authorizing `identity:get_project(project_id=d09dbd1cfa2d418d9489e5d90c58a910)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:37.579180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73717b-3a3f-4891-9452-297a7cb71138 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:37.580887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 592/1184] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:37 2022] GET /identity/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:37.587359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:37.696345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942e942-aca4-4210-a1ba-e4d89cc2acd0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:37.722698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:37.731214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a5a47-c17f-4087-a5db-4e1dc4bdafe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:37.731923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a5a47-c17f-4087-a5db-4e1dc4bdafe8 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:37.732201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a5a47-c17f-4087-a5db-4e1dc4bdafe8 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:37.732482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a5a47-c17f-4087-a5db-4e1dc4bdafe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:37.733302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a5a47-c17f-4087-a5db-4e1dc4bdafe8 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=d09dbd1cfa2d418d9489e5d90c58a910, user.id=77bda763d13048749354bf5c1ae1d59a) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:37.733821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a5a47-c17f-4087-a5db-4e1dc4bdafe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:37.735260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a5a47-c17f-4087-a5db-4e1dc4bdafe8 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:37.740875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 593/1185] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:40:37 2022] GET /identity/v3/role_assignments?user.id=77bda763d13048749354bf5c1ae1d59a&scope.project.id=d09dbd1cfa2d418d9489e5d90c58a910&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:39.929509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a8567-5352-4f9b-b1fa-e542d25d49df None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:39.930426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a8567-5352-4f9b-b1fa-e542d25d49df None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:39.930426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a8567-5352-4f9b-b1fa-e542d25d49df None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:39.932056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 593/1186] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:39.939255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4d89c-d557-4ee2-b7cc-378bb13188de None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:39.940025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4d89c-d557-4ee2-b7cc-378bb13188de None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:39.940212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4d89c-d557-4ee2-b7cc-378bb13188de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:39.962296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-aff4d89c-d557-4ee2-b7cc-378bb13188de None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:39.963544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-aff4d89c-d557-4ee2-b7cc-378bb13188de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:40.054122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 594/1187] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:40.082264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a8567-5352-4f9b-b1fa-e542d25d49df None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:40.210810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a8567-5352-4f9b-b1fa-e542d25d49df None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:40.248125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:40.248929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:40.249214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:40.249490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:40.261154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:40:40.268852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:40:40.269995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:40:40.271149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 594/1188] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:40:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:40:40.281089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4d89c-d557-4ee2-b7cc-378bb13188de None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:40.394361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4d89c-d557-4ee2-b7cc-378bb13188de None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:40.420528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:40.440452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:40.441549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:40.442056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:40.442552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:40.443872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:40.444277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:40.445777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:40.454831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 595/1189] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:40:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:40.461416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:40.517193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:40.585965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057479a8-4416-4ee6-b38d-c1901d81bdca None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:40.627794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:40.628659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:40.628935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:40.629202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] PATH_INFO: `/v3/users/77bda763d13048749354bf5c1ae1d59a` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:40.630429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] RBAC: Authorizing `identity:get_user(user_id=77bda763d13048749354bf5c1ae1d59a)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:40.632437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:40.634068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 595/1190] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:40 2022] GET /identity/v3/users/77bda763d13048749354bf5c1ae1d59a => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:40.644048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:40.756921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7750d3f-d93a-4d67-8b69-abbd9ed3c4fd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:40.791631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f97d0-5eb6-4f00-8862-71ec443c5a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:40.792497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f97d0-5eb6-4f00-8862-71ec443c5a35 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:40.792789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f97d0-5eb6-4f00-8862-71ec443c5a35 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:40.793068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f97d0-5eb6-4f00-8862-71ec443c5a35 None admin] PATH_INFO: `/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:40.794296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f97d0-5eb6-4f00-8862-71ec443c5a35 None admin] RBAC: Authorizing `identity:get_project(project_id=d09dbd1cfa2d418d9489e5d90c58a910)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:40.795768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f97d0-5eb6-4f00-8862-71ec443c5a35 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:40.797837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 596/1191] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:40 2022] GET /identity/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:40.805043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:40.924448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83012635-fbb8-408e-9708-84919f9a9101 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:40.933304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:40.962941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2125a-b98d-44d5-8b72-a2690b3fcd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:40.963867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2125a-b98d-44d5-8b72-a2690b3fcd7e None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:40.964149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2125a-b98d-44d5-8b72-a2690b3fcd7e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:40.964427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2125a-b98d-44d5-8b72-a2690b3fcd7e None admin] PATH_INFO: `/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910/users/77bda763d13048749354bf5c1ae1d59a/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:40.966486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2125a-b98d-44d5-8b72-a2690b3fcd7e None admin] RBAC: Authorizing `identity:create_grant(project_id=d09dbd1cfa2d418d9489e5d90c58a910, user_id=77bda763d13048749354bf5c1ae1d59a, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:40.967969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2125a-b98d-44d5-8b72-a2690b3fcd7e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:40.987484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 596/1192] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:40:40 2022] PUT /identity/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910/users/77bda763d13048749354bf5c1ae1d59a/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:40:43.057956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476afa-db81-4a8d-86cf-b7c9098d03c7 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:43.058851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476afa-db81-4a8d-86cf-b7c9098d03c7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:43.058851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476afa-db81-4a8d-86cf-b7c9098d03c7 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:43.061093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 597/1193] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:40:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:40:43.070023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535d1d-c930-44f8-a8a4-e74d26326dd9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:43.070458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535d1d-c930-44f8-a8a4-e74d26326dd9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:43.070585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535d1d-c930-44f8-a8a4-e74d26326dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:43.093457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-28535d1d-c930-44f8-a8a4-e74d26326dd9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:40:43.094647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-28535d1d-c930-44f8-a8a4-e74d26326dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:40:43.167772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 597/1194] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:40:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:40:43.192762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476afa-db81-4a8d-86cf-b7c9098d03c7 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:43.254193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:43.304766 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476afa-db81-4a8d-86cf-b7c9098d03c7 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:43.344126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:43.344866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:43.345151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:43.345426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:43.356893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:40:43.364564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:40:43.365607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:40:43.366715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 598/1195] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:40:43.374842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535d1d-c930-44f8-a8a4-e74d26326dd9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:43.471040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:43.476500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535d1d-c930-44f8-a8a4-e74d26326dd9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:43.514183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:43.514846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:43.515130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:43.515432 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:43.516181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:43.516488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:43.517887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:43.526352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 598/1196] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:43.533559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:43.638505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe07b67-b5f1-441a-bcf3-2bb1911862dd None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:43.651606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:43.674448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:43.675325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:43.675644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:43.675914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] PATH_INFO: `/v3/users/77bda763d13048749354bf5c1ae1d59a` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:43.677159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] RBAC: Authorizing `identity:get_user(user_id=77bda763d13048749354bf5c1ae1d59a)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:43.678794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:43.680469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 599/1197] 10.222.0.87 () {56 vars in 1177 bytes} [Fri Jun 17 12:40:43 2022] GET /identity/v3/users/77bda763d13048749354bf5c1ae1d59a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:43.690162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:43.797862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496ac55-d425-4cd2-82b7-81291f083f78 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:43.835187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:43.835875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:43.836160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:43.836435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:43.864842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:43.906475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:43.907901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:44.011119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 599/1198] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:40:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:40:44.017364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:44.125463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6456903-1aab-459c-a08b-fbb7fa3d6309 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:44.165673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d4c73-be95-4b08-8750-a7d8d7ef661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:44.166457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d4c73-be95-4b08-8750-a7d8d7ef661a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:44.166734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d4c73-be95-4b08-8750-a7d8d7ef661a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:44.167003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d4c73-be95-4b08-8750-a7d8d7ef661a None admin] PATH_INFO: `/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:44.168314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d4c73-be95-4b08-8750-a7d8d7ef661a None admin] RBAC: Authorizing `identity:get_project(project_id=d09dbd1cfa2d418d9489e5d90c58a910)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:44.169747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d4c73-be95-4b08-8750-a7d8d7ef661a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:44.171472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 600/1199] 10.222.0.87 () {56 vars in 1186 bytes} [Fri Jun 17 12:40:44 2022] GET /identity/v3/projects/d09dbd1cfa2d418d9489e5d90c58a910 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:40:44.178522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:40:44.246541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:44.289730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf635-65d0-4bdc-81c5-445f6302e412 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:40:44.324708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c14c67-e15a-4537-b385-165d046add1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:40:44.325366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14c67-e15a-4537-b385-165d046add1a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:40:44.325647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14c67-e15a-4537-b385-165d046add1a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:40:44.325923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14c67-e15a-4537-b385-165d046add1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:40:44.326765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c14c67-e15a-4537-b385-165d046add1a None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=d09dbd1cfa2d418d9489e5d90c58a910, user.id=77bda763d13048749354bf5c1ae1d59a) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:40:44.327083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c14c67-e15a-4537-b385-165d046add1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:40:44.328565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c14c67-e15a-4537-b385-165d046add1a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:40:44.334495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 600/1200] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:40:44 2022] GET /identity/v3/role_assignments?user.id=77bda763d13048749354bf5c1ae1d59a&scope.project.id=d09dbd1cfa2d418d9489e5d90c58a910&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:40:54.160910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=76 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:40:54.331735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=38 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:41.560300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f1e6f-d90d-458c-9811-62e5c5151c5c None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:41.560920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f1e6f-d90d-458c-9811-62e5c5151c5c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:41.560920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f1e6f-d90d-458c-9811-62e5c5151c5c None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:41.562702 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 601/1201] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:41:41.569530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb14c-9916-4d1d-beb8-3421e57050b1 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:41.569530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb14c-9916-4d1d-beb8-3421e57050b1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:41.569530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb14c-9916-4d1d-beb8-3421e57050b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:41.589339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a49cb14c-9916-4d1d-beb8-3421e57050b1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:41:41.591335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a49cb14c-9916-4d1d-beb8-3421e57050b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:41:41.685051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 601/1202] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:41:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:41:41.710584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f1e6f-d90d-458c-9811-62e5c5151c5c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:41.848245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f1e6f-d90d-458c-9811-62e5c5151c5c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:41.891244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:41.892127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:41.892457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:41.892803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:41.904953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:41.906590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:41.927342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:41.928447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:41.929887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:41.929887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:41.929887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:41.929887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:41.929887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 602/1203] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:41.935148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb14c-9916-4d1d-beb8-3421e57050b1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:42.051582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb14c-9916-4d1d-beb8-3421e57050b1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:42.087344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:42.094788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8accfa9-5790-45cc-8d2e-0521475e5b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:42.095620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8accfa9-5790-45cc-8d2e-0521475e5b4b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:42.096063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8accfa9-5790-45cc-8d2e-0521475e5b4b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:42.096341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8accfa9-5790-45cc-8d2e-0521475e5b4b None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:42.097307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8accfa9-5790-45cc-8d2e-0521475e5b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:42.097532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8accfa9-5790-45cc-8d2e-0521475e5b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:42.099172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8accfa9-5790-45cc-8d2e-0521475e5b4b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:42.118623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 602/1204] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:41:42.124539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:42.185856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:42.238298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffabf7b-fa12-437e-92dc-8d944c8f7a18 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:42.285764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92034df6-0382-4f13-bcd7-216062499f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:42.286495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92034df6-0382-4f13-bcd7-216062499f61 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:42.286773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92034df6-0382-4f13-bcd7-216062499f61 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:42.287054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92034df6-0382-4f13-bcd7-216062499f61 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:42.287848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92034df6-0382-4f13-bcd7-216062499f61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:42.289357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92034df6-0382-4f13-bcd7-216062499f61 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:42.317214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 603/1205] 10.222.0.87 () {60 vars in 1132 bytes} [Fri Jun 17 12:41:42 2022] POST /identity/v3/users => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:41:44.779747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea6338-782d-4e2a-b801-876a2d6136bc None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:44.780394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea6338-782d-4e2a-b801-876a2d6136bc None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:44.780394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea6338-782d-4e2a-b801-876a2d6136bc None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:44.782260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 603/1206] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:41:44.788877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931f73f-0ef7-439e-8d38-dcde0b8be6fe None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:44.789323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931f73f-0ef7-439e-8d38-dcde0b8be6fe None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:44.789449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931f73f-0ef7-439e-8d38-dcde0b8be6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:44.808123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8931f73f-0ef7-439e-8d38-dcde0b8be6fe None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:41:44.809699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8931f73f-0ef7-439e-8d38-dcde0b8be6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:41:44.897111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 604/1207] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:41:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:41:44.921704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea6338-782d-4e2a-b801-876a2d6136bc None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:45.039307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea6338-782d-4e2a-b801-876a2d6136bc None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:45.079551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:45.080390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:45.080674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:45.080949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:45.097935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:41:45.106275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:45.107416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:45.108275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 604/1208] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:41:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:45.119157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931f73f-0ef7-439e-8d38-dcde0b8be6fe None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:45.164265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:45.237208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931f73f-0ef7-439e-8d38-dcde0b8be6fe None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:45.276680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:45.277489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:45.277833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:45.278114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:45.279039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:45.279385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:45.281019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:45.291467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 605/1209] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:41:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:45.303181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:45.361783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:45.419871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb8d92-1ddb-48cf-8b5e-6c430f682f74 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:45.461711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:45.462662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:45.462982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:45.463293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:45.475698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:45.477400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:45.499449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:45.502512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:45.503748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:45.503748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:45.503748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:45.503748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:45.503748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 605/1210] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:45.506558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:45.621524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:45.625000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a48a-3599-4bc7-a106-030dfa9a3547 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:45.663053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:45.663763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:45.664053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:45.664333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:45.665065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:45.665377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:45.666750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:45.680568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 606/1211] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:41:45.686599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:45.800730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ff86d-d1fb-47ac-9530-1653c9302068 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:45.802749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:45.844071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:45.844843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:45.845129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:45.845406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:45.855824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:45.857303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:41:45.872986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:45.874035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:45.879751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 606/1212] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:45.880760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:45.997516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b731-7271-4a36-aac7-28c559ea1e96 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:46.042361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:46.043228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:46.043568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:46.043845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:46.044706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:46.045021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:46.046468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:46.071257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 607/1213] 10.222.0.87 () {56 vars in 1143 bytes} [Fri Jun 17 12:41:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:46.073005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:46.182976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17385242-f91e-4774-b87b-0d084cb4e5d5 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:46.231267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:46.231990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:46.232279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:46.232557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:46.269344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:46.314371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:46.316041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:46.430729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 607/1214] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:41:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:41:46.437996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:46.467983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:46.561114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673887be-f862-49ff-98b6-e6f597bbf385 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:46.602653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:46.603913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:46.604450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:46.605105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:46.617618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:46.619569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:46.645763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:46.646892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:46.647993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:46.647993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:46.647993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:46.647993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:46.649693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 608/1215] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:46.656342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:46.725508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:46.775129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe46be-32a4-40a5-a7d3-11126fdc10d4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:46.813041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:46.813851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:46.814144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:46.814552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:46.815284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:46.815627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:46.817022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:46.830032 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 608/1216] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:46.836651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:46.917558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:46.957899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa9537-c286-4709-8100-b3bbc0dac87a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:47.004506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:47.005678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:47.006104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:47.006535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:47.015847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:47.018095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:41:47.037325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:47.038354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:47.044528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 609/1217] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:41:47.047014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:47.156941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa25da-53d0-4a64-99d8-dc69998e2778 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:47.183116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:47.196685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c69fab-e4fd-41a6-a10e-bc3d74928405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:47.197357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c69fab-e4fd-41a6-a10e-bc3d74928405 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:47.197637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c69fab-e4fd-41a6-a10e-bc3d74928405 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:47.197911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c69fab-e4fd-41a6-a10e-bc3d74928405 None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:47.198787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c69fab-e4fd-41a6-a10e-bc3d74928405 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:47.199097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c69fab-e4fd-41a6-a10e-bc3d74928405 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:47.200576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c69fab-e4fd-41a6-a10e-bc3d74928405 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:47.218592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 609/1218] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:41:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:47.220927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:47.340264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4c86b-75c4-4ce2-ad8c-83e93a788104 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:47.386618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:47.393202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9d3ae-ca39-482d-a19e-8e1e822f9fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:47.393952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9d3ae-ca39-482d-a19e-8e1e822f9fcb None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:47.394311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9d3ae-ca39-482d-a19e-8e1e822f9fcb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:47.394628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9d3ae-ca39-482d-a19e-8e1e822f9fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:47.395601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9d3ae-ca39-482d-a19e-8e1e822f9fcb None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=e34865acbc294b3b851bcea0cf6b442e) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:47.395972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9d3ae-ca39-482d-a19e-8e1e822f9fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:47.397488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9d3ae-ca39-482d-a19e-8e1e822f9fcb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:47.415407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 610/1219] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:41:47 2022] GET /identity/v3/role_assignments?user.id=e34865acbc294b3b851bcea0cf6b442e&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:49.624545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac3fd-516a-4328-b15d-7e11c2f4ba28 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:49.625171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac3fd-516a-4328-b15d-7e11c2f4ba28 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:49.625171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac3fd-516a-4328-b15d-7e11c2f4ba28 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:49.626672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 610/1220] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:41:49.632192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128635f-8b5b-4f1c-8180-b3c371b2aa91 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:49.632644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128635f-8b5b-4f1c-8180-b3c371b2aa91 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:49.632771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128635f-8b5b-4f1c-8180-b3c371b2aa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:49.650345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8128635f-8b5b-4f1c-8180-b3c371b2aa91 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:41:49.651591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8128635f-8b5b-4f1c-8180-b3c371b2aa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:41:49.731865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 611/1221] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:41:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:41:49.754782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac3fd-516a-4328-b15d-7e11c2f4ba28 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:49.877481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac3fd-516a-4328-b15d-7e11c2f4ba28 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:49.916180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:49.916987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:49.917510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:49.917759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:49.928485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:41:49.936508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:49.938075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:49.939263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 611/1222] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:41:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:49.944278 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128635f-8b5b-4f1c-8180-b3c371b2aa91 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:50.073739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128635f-8b5b-4f1c-8180-b3c371b2aa91 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:50.098341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:50.110596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:50.111284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:50.111589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:50.111872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:50.112635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] RBAC: Adding query filter params (name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:50.112946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:50.114318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:50.123623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 612/1223] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:41:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:50.129990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:50.176457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:50.238340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a94c6c-4e9d-4eb1-a90a-a3aea7d5e7a4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:50.276492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:50.277451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:50.277744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:50.278018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:50.291019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:50.292729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:50.310126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:50.311207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:50.312256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:50.312256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:50.312256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:50.312256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:50.312256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 612/1224] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:50.317566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:50.442684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d5310-5188-40f8-aae7-0dd3050ca479 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:50.481777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:50.482461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:50.482750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:50.483052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:50.483823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:50.484114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:50.485491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:50.499666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 613/1225] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:50.505400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:50.605665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:50.613106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfd471-6ba4-445c-b813-d036300ae90b None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:50.650423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:50.651253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:50.651571 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:50.651891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:50.663400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:50.665010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:41:50.682228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:50.683772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:50.684723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 613/1226] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:50.689994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:50.755001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:50.820419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026531bd-55d9-4b8e-bf69-02a41b63cfd7 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:50.866035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:50.867041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:50.867554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:50.867965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:50.869053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:50.869514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:50.871931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:50.893999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 614/1227] 10.222.0.87 () {56 vars in 1143 bytes} [Fri Jun 17 12:41:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:50.900258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:51.012636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbf1d6-42a9-4e36-813b-141a4996fcd4 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:51.050878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:51.052506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:51.052989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:51.053454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:51.067786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:51.069423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:51.090589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:51.091854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:51.092861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:51.092861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:51.092861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:51.092861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:51.092861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 614/1228] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:51.099159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:51.222731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26589319-9fc9-4b18-8be8-e47645290901 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:51.225290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:51.258573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:51.259246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:51.259564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:51.259844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:51.260571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:51.260884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:51.262682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:51.274142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 615/1229] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:41:51.280366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:51.290725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:51.395111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723600f7-a34b-452a-89fc-a3cc55550acd None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:51.432215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:51.433055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:51.433342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:51.433618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:51.440594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:51.442057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:41:51.457765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:51.458879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:51.464836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 615/1230] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:51.465836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:51.587818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c3cf7-af85-4688-b51e-0b4077e6a2ae None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:51.628102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5cb64-0252-4b0b-b9de-efa9f9b89838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:51.628809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5cb64-0252-4b0b-b9de-efa9f9b89838 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:51.629113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5cb64-0252-4b0b-b9de-efa9f9b89838 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:51.629467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5cb64-0252-4b0b-b9de-efa9f9b89838 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:51.630225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5cb64-0252-4b0b-b9de-efa9f9b89838 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:51.630547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5cb64-0252-4b0b-b9de-efa9f9b89838 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:51.632091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5cb64-0252-4b0b-b9de-efa9f9b89838 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:51.645943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 616/1231] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:51.655095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:51.725491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:51.766950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452d457-d423-487d-94c2-a7aef45ab186 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:51.806240 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0952350-6a58-4c78-a191-59530e4b8d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:51.807270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952350-6a58-4c78-a191-59530e4b8d58 None admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:51.807624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952350-6a58-4c78-a191-59530e4b8d58 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:51.807898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952350-6a58-4c78-a191-59530e4b8d58 None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/e34865acbc294b3b851bcea0cf6b442e/roles/e381464c740146119ff513d42bb0044b` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:51.827595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0952350-6a58-4c78-a191-59530e4b8d58 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=e34865acbc294b3b851bcea0cf6b442e, role_id=e381464c740146119ff513d42bb0044b)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:51.829219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0952350-6a58-4c78-a191-59530e4b8d58 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:51.843520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 616/1232] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:41:51 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/e34865acbc294b3b851bcea0cf6b442e/roles/e381464c740146119ff513d42bb0044b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:41:54.100377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35630-77b1-4f2f-8c90-61b5982aac4a None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:54.100973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35630-77b1-4f2f-8c90-61b5982aac4a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:54.100973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35630-77b1-4f2f-8c90-61b5982aac4a None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:54.102502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 617/1233] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:41:54.107827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79b18-13f2-4873-8a2c-2c7d3b1bb482 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:54.108309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79b18-13f2-4873-8a2c-2c7d3b1bb482 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:54.108420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79b18-13f2-4873-8a2c-2c7d3b1bb482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:54.127803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9ea79b18-13f2-4873-8a2c-2c7d3b1bb482 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:41:54.129165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9ea79b18-13f2-4873-8a2c-2c7d3b1bb482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:41:54.209493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 617/1234] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:41:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:41:54.236931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35630-77b1-4f2f-8c90-61b5982aac4a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:54.265018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:54.350622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35630-77b1-4f2f-8c90-61b5982aac4a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:54.388693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:54.389564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:54.389875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:54.390204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:54.402108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:41:54.410048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:54.411256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:54.412340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 618/1235] 10.222.0.87 () {56 vars in 1102 bytes} [Fri Jun 17 12:41:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:54.417448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea79b18-13f2-4873-8a2c-2c7d3b1bb482 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:54.466834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:54.523999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea79b18-13f2-4873-8a2c-2c7d3b1bb482 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:54.561948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:54.562920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:54.563316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:54.563753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:54.564673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] RBAC: Adding query filter params (name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:54.565085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:54.566991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:54.585677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 618/1236] 10.222.0.87 () {56 vars in 1103 bytes} [Fri Jun 17 12:41:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:54.586990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:54.682560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:54.702005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76eea1-fd37-4713-b06d-b1f8ae3fc7d0 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:54.740176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:54.740970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:54.741256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:54.741532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:54.753804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:54.755580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:54.774276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:54.775331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:54.776271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:54.776271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:54.776271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:54.776271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:54.776271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 619/1237] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:54.782315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:54.899056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c2554-05b5-48d4-bae4-247e6b3801fb None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:54.909301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:54.937246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:54.937853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:54.938164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:54.938495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:54.939286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:54.939629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:54.941266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:54.958149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 619/1238] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:54.964087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:55.078452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb69cd-eee9-4940-a96e-5a4bc870a437 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:55.116692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:55.117623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:55.117902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:55.118151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:55.121750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:55.128193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:55.129843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:41:55.145347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:55.146352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:55.148743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 620/1239] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:55.152976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:55.264812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19d04a-e5eb-4bac-9c86-49b9423666b3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:55.301779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:55.302478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:55.302713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:55.303005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:55.303867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:55.304176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:55.305583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:55.329660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 620/1240] 10.222.0.87 () {56 vars in 1143 bytes} [Fri Jun 17 12:41:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:55.330948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:55.447674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0f7fc-5ae9-42da-bb65-a883358dfb77 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:55.485757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:55.486482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:55.486799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:55.487079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:55.555869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:55.563472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:55.565230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:55.679400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 621/1241] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:41:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:41:55.683363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:55.714050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:55.797746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bd9f9-c579-4222-965d-0dfae9ad4258 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:55.840292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:55.841527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:55.841905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:55.842272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:55.856160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:55.857953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:55.879503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:55.880654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:55.881652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:55.881652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:55.881652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:55.881652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:55.881652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 621/1242] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:55.886929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:55.988605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:55.994915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ff30c-7b90-467e-a6d5-1b1f9cf6044f None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:56.030796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:56.031540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:56.031832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:56.032114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:56.032854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:56.033167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:56.034733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:56.048593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 622/1243] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:41:56.054655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:56.150425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:56.174514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe9fbe-e878-4f05-8535-bcf1aff59c54 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:56.212094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:56.212959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:56.213248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:56.213532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:56.220876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:56.222328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:41:56.235871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:56.236841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:41:56.237802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 622/1244] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:56.243479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:56.354683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c99c50-283d-4cb9-88cc-0bbf9b0bec3a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:56.397165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffd195-3fae-4560-803e-fbad4380ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:56.397765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd195-3fae-4560-803e-fbad4380ff97 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:56.398049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd195-3fae-4560-803e-fbad4380ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:56.398414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd195-3fae-4560-803e-fbad4380ff97 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:56.399293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffd195-3fae-4560-803e-fbad4380ff97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:56.399711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffd195-3fae-4560-803e-fbad4380ff97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:56.401353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffd195-3fae-4560-803e-fbad4380ff97 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:56.415050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 623/1245] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:56.421691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:56.534712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faeca43-5769-4b53-949e-794ecacbbaa0 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:56.571730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91a853-2ffb-4216-ab0f-eb5d9cede542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:56.572480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91a853-2ffb-4216-ab0f-eb5d9cede542 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:56.572837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91a853-2ffb-4216-ab0f-eb5d9cede542 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:56.573120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91a853-2ffb-4216-ab0f-eb5d9cede542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:56.574309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91a853-2ffb-4216-ab0f-eb5d9cede542 None admin] RBAC: Adding query filter params (role.id=e381464c740146119ff513d42bb0044b, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=e34865acbc294b3b851bcea0cf6b442e) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:56.574676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91a853-2ffb-4216-ab0f-eb5d9cede542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:56.576318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91a853-2ffb-4216-ab0f-eb5d9cede542 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:56.582720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 623/1246] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:41:56 2022] GET /identity/v3/role_assignments?user.id=e34865acbc294b3b851bcea0cf6b442e&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=e381464c740146119ff513d42bb0044b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:58.869660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e29b0-7c77-494a-8124-4b061b135879 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:58.870327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e29b0-7c77-494a-8124-4b061b135879 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:58.870327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e29b0-7c77-494a-8124-4b061b135879 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:58.872379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 624/1247] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:41:58.877829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48e742-8ddc-4ead-9940-bc83d9287638 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:58.878317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48e742-8ddc-4ead-9940-bc83d9287638 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:58.878419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48e742-8ddc-4ead-9940-bc83d9287638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:58.884368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:58.898642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3e48e742-8ddc-4ead-9940-bc83d9287638 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:41:58.900159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3e48e742-8ddc-4ead-9940-bc83d9287638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:41:58.982208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 624/1248] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:41:59.007542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e29b0-7c77-494a-8124-4b061b135879 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:59.028334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:59.131163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e29b0-7c77-494a-8124-4b061b135879 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:59.171273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:59.172166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:59.172530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:59.172783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:59.185388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:41:59.193672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:59.194974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:41:59.196086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:59.201007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 625/1249] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:41:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:41:59.202149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48e742-8ddc-4ead-9940-bc83d9287638 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:59.313443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48e742-8ddc-4ead-9940-bc83d9287638 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:59.348695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:59.354411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:59.355835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:59.356313 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:59.356790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:59.358238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:59.358743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:59.360969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:59.371948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 625/1250] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:59.377649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:59.467628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:59.494542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386dfba-f04a-4ccd-8955-52e378f17d08 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:59.532936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:59.533746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:59.534029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:59.534304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:59.546793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:59.548554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:41:59.568442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:59.569637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:59.575565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:41:59.575565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:41:59.575565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:41:59.575565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:59.575565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 626/1251] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:41:59.576121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:59.681764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45c230-ccb5-45a6-ac56-dfaee6f4523a None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:59.718196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:59.718793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:59.719080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:59.719351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:59.720115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:41:59.720425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:59.721807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:59.734432 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 626/1252] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:41:59.740406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:41:59.864513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07228b-4f20-4854-94e9-e8a98c6816e4 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:41:59.894893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:41:59.904576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:41:59.905488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:41:59.905788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:41:59.906138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] PATH_INFO: `/v3/users/placement` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:41:59.914704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:41:59.916395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:41:59.932976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:41:59.934175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:41:59.935129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 627/1253] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:41:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:41:59.940503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:00.003754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:00.052207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe2a70-c732-44df-95a8-fb1de0a9e58d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:00.091193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:00.091983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:00.092296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:00.092596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:00.093500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:00.093816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:00.095269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:00.116311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 627/1254] 10.222.0.87 () {56 vars in 1143 bytes} [Fri Jun 17 12:41:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:00.123883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:00.260419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a509c-e239-4632-a92c-e928ed7f033d None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:00.303630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:00.304293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:00.304590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:00.304865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:00.360476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:00.373464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:00.375059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:00.486268 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 628/1255] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:42:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:42:00.492642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:00.600515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea26ac-6661-4a41-a3e7-ef1b9257ddf0 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:00.614332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:00.636147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:00.637220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:00.637511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:00.637788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:00.649200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:00.650673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:42:00.669166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:00.670309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:00.672298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:42:00.672298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:42:00.672298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:00.672298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:00.672298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 628/1256] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:00.676529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:00.771239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:00.785615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd803f-a607-4352-be70-79e3009423bb None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:00.824342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:00.825050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:00.825371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:00.825651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:00.826503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:00.826846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:00.828382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:00.841649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 629/1257] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:00.848324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:00.957485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6984c9-98eb-47be-983c-0160e7365e20 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:00.993129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:00.993979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:00.994287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:00.994571 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] PATH_INFO: `/v3/projects/service` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:01.001783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:01.003288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:42:01.017290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:01.018315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:01.019227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 629/1258] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:42:01.025524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:01.136673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fc107-170f-49f1-91f9-7e2139d5a385 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:01.176105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c495cc0-ce9c-425a-a820-2309e54e0558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:01.177199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495cc0-ce9c-425a-a820-2309e54e0558 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:01.177510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495cc0-ce9c-425a-a820-2309e54e0558 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:01.177787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495cc0-ce9c-425a-a820-2309e54e0558 None admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:01.178614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c495cc0-ce9c-425a-a820-2309e54e0558 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:01.178919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c495cc0-ce9c-425a-a820-2309e54e0558 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:01.180416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c495cc0-ce9c-425a-a820-2309e54e0558 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:01.193445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 630/1259] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:01.199877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:01.227894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:01.312863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5e44d-e3e7-4195-8005-2cf04beaabed None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:01.348131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d49e0-5bde-47b0-b6c5-161ce774187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:01.348863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d49e0-5bde-47b0-b6c5-161ce774187d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:01.349148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d49e0-5bde-47b0-b6c5-161ce774187d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:01.349424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d49e0-5bde-47b0-b6c5-161ce774187d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:01.350275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d49e0-5bde-47b0-b6c5-161ce774187d None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=e34865acbc294b3b851bcea0cf6b442e) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:01.350590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d49e0-5bde-47b0-b6c5-161ce774187d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:01.352079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d49e0-5bde-47b0-b6c5-161ce774187d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:01.358586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 630/1260] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:42:01 2022] GET /identity/v3/role_assignments?user.id=e34865acbc294b3b851bcea0cf6b442e&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=4a6b9a3327544100929540645a797d97 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:42:03.471769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a2e4f-f621-4ef7-aa5d-d35ffb9a113b None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:03.472573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a2e4f-f621-4ef7-aa5d-d35ffb9a113b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:03.472573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a2e4f-f621-4ef7-aa5d-d35ffb9a113b None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:03.474761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 631/1261] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:42:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:42:03.480259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35f4aa-c86b-429d-bf8c-0b26547d6c0d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:03.480884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35f4aa-c86b-429d-bf8c-0b26547d6c0d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:03.480884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35f4aa-c86b-429d-bf8c-0b26547d6c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:03.506798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ab35f4aa-c86b-429d-bf8c-0b26547d6c0d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:42:03.508799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ab35f4aa-c86b-429d-bf8c-0b26547d6c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:42:03.584551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 631/1262] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:42:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:42:03.609233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a2e4f-f621-4ef7-aa5d-d35ffb9a113b None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:03.620358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:03.718328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a2e4f-f621-4ef7-aa5d-d35ffb9a113b None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:03.753245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:03.754035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:03.754316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:03.754586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:03.765607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:42:03.773201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:03.774144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:42:03.774992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 632/1263] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:42:03.781062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35f4aa-c86b-429d-bf8c-0b26547d6c0d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:03.800854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:03.886527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35f4aa-c86b-429d-bf8c-0b26547d6c0d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:03.929096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:03.929777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:03.930066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:03.930346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:03.931103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] RBAC: Adding query filter params (name=admin) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:03.931438 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:03.932829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:03.941372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 632/1264] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:03.947464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:04.016393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:04.051501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f21be-cd5e-4119-94d1-a8db8800b1d1 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:04.087540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:04.088335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:04.088619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:04.088894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:04.100441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:04.101900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:42:04.119480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:04.120486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:04.122436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:42:04.122436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:42:04.122436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:04.122436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:42:04.122436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 633/1265] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:04.127519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:04.223917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:04.242823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4faf67-cab4-4a67-b41e-566246160797 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:04.275359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:04.275991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:04.276277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:04.276556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:04.277277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:04.277605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:04.278981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:04.290303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 633/1266] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:42:04.298969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:04.406927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a25b34-9c72-42e5-bf57-9a945f89fceb None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:04.426611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:04.442017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:04.442821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:04.443111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:04.443399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:04.451276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:04.452761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:42:04.467292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:04.468379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:42:04.470298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 634/1267] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:42:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:04.480677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:04.588293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa16125-3ef6-4679-953b-5ed25b721538 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:04.624169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:04.624822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:04.625101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:04.625361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:04.626191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:04.626498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:04.627967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:04.637368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:04.652368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 634/1268] 10.222.0.87 () {56 vars in 1143 bytes} [Fri Jun 17 12:42:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:42:04.658707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:04.766946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430c39f-f21b-4ff2-a79c-dde0d2c6af34 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:04.806035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:04.806821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:04.807107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:04.807422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:04.818323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:04.819862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:42:04.837412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:04.838577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:04.839737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:42:04.839737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:42:04.839737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:04.839737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:42:04.839737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 635/1269] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:04.845714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:04.952120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d985-0800-434f-ae06-febb220b70d9 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:04.987300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:04.987998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:04.988282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:04.988561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:04.989292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:04.989601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:04.990983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:05.002273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 635/1270] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:05.011428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:05.046128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:05.119885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0ee0e-c407-429d-837a-9a0cce9194f6 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:05.158016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:05.158808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:05.159091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:05.159364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:05.166847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:05.168449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:42:05.183616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:05.184801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:42:05.185779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 636/1271] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:05.193459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:05.263228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:05.312325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df57b6f-06ad-478b-a054-f83af98883c3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:05.348248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176c0aa-cdab-439b-babb-1768b0b89e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:05.349086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176c0aa-cdab-439b-babb-1768b0b89e2b None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:05.349374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176c0aa-cdab-439b-babb-1768b0b89e2b None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:05.349653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176c0aa-cdab-439b-babb-1768b0b89e2b None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:05.350443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176c0aa-cdab-439b-babb-1768b0b89e2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:05.350752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176c0aa-cdab-439b-babb-1768b0b89e2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:05.352228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176c0aa-cdab-439b-babb-1768b0b89e2b None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:05.365234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 636/1272] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:42:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:05.374423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:05.478983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:05.489145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3f96c-4ef9-4f91-89e1-3eba0fd31a6e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:05.525006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b6d1d-bd7e-4322-9963-73b811a5c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:05.525883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b6d1d-bd7e-4322-9963-73b811a5c72e None admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:05.526164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b6d1d-bd7e-4322-9963-73b811a5c72e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:05.526439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b6d1d-bd7e-4322-9963-73b811a5c72e None admin] PATH_INFO: `/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/e34865acbc294b3b851bcea0cf6b442e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:05.528804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b6d1d-bd7e-4322-9963-73b811a5c72e None admin] RBAC: Authorizing `identity:create_grant(project_id=68f70dff20b8427180b9b3b5a8b418cd, user_id=e34865acbc294b3b851bcea0cf6b442e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:05.530237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b6d1d-bd7e-4322-9963-73b811a5c72e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:05.547425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 637/1273] 10.222.0.87 () {58 vars in 1439 bytes} [Fri Jun 17 12:42:05 2022] PUT /identity/v3/projects/68f70dff20b8427180b9b3b5a8b418cd/users/e34865acbc294b3b851bcea0cf6b442e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:42:07.714512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31f3a-6055-4895-acbe-4d24fb0bc8a4 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:07.715483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31f3a-6055-4895-acbe-4d24fb0bc8a4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:07.715483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31f3a-6055-4895-acbe-4d24fb0bc8a4 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:07.717722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 637/1274] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:42:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:42:07.724593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da941fa0-0655-42db-87ea-7fecccb0a2c3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:07.725352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da941fa0-0655-42db-87ea-7fecccb0a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:07.725518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da941fa0-0655-42db-87ea-7fecccb0a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:07.745354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-da941fa0-0655-42db-87ea-7fecccb0a2c3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:42:07.746554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-da941fa0-0655-42db-87ea-7fecccb0a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:42:07.821533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 638/1275] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:42:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:42:07.861443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31f3a-6055-4895-acbe-4d24fb0bc8a4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:07.971946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31f3a-6055-4895-acbe-4d24fb0bc8a4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:07.989674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:08.008806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:08.009612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:08.009897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:08.010172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:08.020948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 12:42:08.028583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:08.029641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:08.030501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 638/1276] 10.222.0.87 () {56 vars in 1096 bytes} [Fri Jun 17 12:42:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:08.037981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da941fa0-0655-42db-87ea-7fecccb0a2c3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:08.122607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:08.142289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da941fa0-0655-42db-87ea-7fecccb0a2c3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:08.176679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:08.177712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:08.177999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:08.178275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:08.179037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:08.179343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:08.180750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:08.189394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 639/1277] 10.222.0.87 () {56 vars in 1099 bytes} [Fri Jun 17 12:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:08.196279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:08.309854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d96e5-c1b7-42aa-81c3-3de7897d87fc None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:08.349307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:08.350108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:08.350389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:08.350651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:08.362658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:08.364134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:42:08.389828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:08.391773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:08.393497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:42:08.393497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:42:08.393497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:08.393497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:08.393497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 639/1278] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:08.402072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:08.510692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff919e8-a6c6-4f97-8ec1-db025c392b6e None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:08.545511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:08.546172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:08.546455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:08.546740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:08.547495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:08.547787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:08.549161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:08.560881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 640/1279] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:08.569266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:08.601719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:08.671055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b4093-7b37-4bf6-a760-bf0104362c6d None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:08.706408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:08.707208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:08.707526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:08.707802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] PATH_INFO: `/v3/users/placement` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:08.715526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:08.716976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 12:42:08.731493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:08.732555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:08.733873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 640/1280] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:42:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 12:42:08.738635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:08.742469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:08.846117 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de762-bc10-47bc-a1c5-ba7899b344e6 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:08.881561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:08.882226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:08.882507 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:08.882783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:08.883667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:08.883981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:08.885409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:08.902697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 641/1281] 10.222.0.87 () {56 vars in 1143 bytes} [Fri Jun 17 12:42:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:08.908847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:09.006958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:09.019089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207f843-3de9-49e6-9473-d03ad8b65c4e None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:09.054707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:09.055347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:09.055671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:09.055951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:09.125483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:09.126961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:09.218987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:09.230792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 641/1282] 10.222.0.87 () {58 vars in 1282 bytes} [Fri Jun 17 12:42:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 12:42:09.237124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:09.297231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:09.350976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7af2f-80cf-4528-af96-648a86ade261 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:09.390223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:09.390891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:09.391205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:09.391524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:09.402099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:09.403763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 12:42:09.423794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:09.424818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:09.428624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 12:42:09.428624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 12:42:09.428624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 12:42:09.428624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:42:09.428624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 642/1283] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:09.433082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:09.538823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff4c4-a15c-4ef7-a08f-433447fc5543 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:09.575751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:09.576411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:09.576694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:09.576973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:09.577695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:09.578007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:09.579420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:09.590758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 642/1284] 10.222.0.87 () {56 vars in 1109 bytes} [Fri Jun 17 12:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:09.599128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:09.718685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdbd15-85f3-443f-824b-84f23e29da01 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:09.732562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:09.755044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:09.755867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:09.756147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:09.756422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] PATH_INFO: `/v3/projects/service` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:09.763736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:09.765177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 12:42:09.779423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:09.780456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: ERROR keystone.server.flask.application Jun 17 12:42:09.781403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 643/1285] 10.222.0.87 () {56 vars in 1111 bytes} [Fri Jun 17 12:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:09.789581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:09.854206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:09.906221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20555f-c81b-4a31-9669-e57fdc1ef6d7 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:09.944129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77977d3-856d-4c1f-bf29-0ed0a296fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:09.944782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77977d3-856d-4c1f-bf29-0ed0a296fdaf None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:09.945067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77977d3-856d-4c1f-bf29-0ed0a296fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:09.945343 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77977d3-856d-4c1f-bf29-0ed0a296fdaf None admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:09.946137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77977d3-856d-4c1f-bf29-0ed0a296fdaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:09.946449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77977d3-856d-4c1f-bf29-0ed0a296fdaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:09.947928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77977d3-856d-4c1f-bf29-0ed0a296fdaf None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:09.960035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 643/1286] 10.222.0.87 () {56 vars in 1148 bytes} [Fri Jun 17 12:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:09.966749 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:10.078407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780795d-9546-460e-84fe-82908b61d80a None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:10.113231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa72ebc-631e-4c0d-963a-45103796ee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:10.113891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa72ebc-631e-4c0d-963a-45103796ee26 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:10.114136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa72ebc-631e-4c0d-963a-45103796ee26 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:10.114442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa72ebc-631e-4c0d-963a-45103796ee26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:10.115282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa72ebc-631e-4c0d-963a-45103796ee26 None admin] RBAC: Adding query filter params (role.id=4a6b9a3327544100929540645a797d97, scope.project.id=68f70dff20b8427180b9b3b5a8b418cd, user.id=e34865acbc294b3b851bcea0cf6b442e) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:10.115617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa72ebc-631e-4c0d-963a-45103796ee26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:10.117046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa72ebc-631e-4c0d-963a-45103796ee26 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:10.123470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 644/1287] 10.222.0.87 () {56 vars in 1374 bytes} [Fri Jun 17 12:42:09 2022] GET /identity/v3/role_assignments?user.id=e34865acbc294b3b851bcea0cf6b442e&scope.project.id=68f70dff20b8427180b9b3b5a8b418cd&role.id=4a6b9a3327544100929540645a797d97 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:12.237269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd09c6-9373-4b4e-bf1e-8cb9381cc145 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:12.237812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd09c6-9373-4b4e-bf1e-8cb9381cc145 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:12.237812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd09c6-9373-4b4e-bf1e-8cb9381cc145 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:12.240005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 644/1288] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:42:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:42:12.248660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3abcd-3c82-4472-8f40-368bafbd030d None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:12.249054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3abcd-3c82-4472-8f40-368bafbd030d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:12.249281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3abcd-3c82-4472-8f40-368bafbd030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:12.272219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-97a3abcd-3c82-4472-8f40-368bafbd030d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:42:12.273425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-97a3abcd-3c82-4472-8f40-368bafbd030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:42:12.305743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:12.353927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 645/1289] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:42:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:42:12.390026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd09c6-9373-4b4e-bf1e-8cb9381cc145 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:12.517575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd09c6-9373-4b4e-bf1e-8cb9381cc145 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:12.528511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:12.555727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:12.556530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:12.556818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:12.557095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] PATH_INFO: `/v3/services/placement` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:12.557759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:12.559137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:12.571280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:12.572518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:12.573378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 645/1290] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:42:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:12.580483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3abcd-3c82-4472-8f40-368bafbd030d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:12.694214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3abcd-3c82-4472-8f40-368bafbd030d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:12.729122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6c00b-34b3-4a78-8136-af17e92293c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:12.729789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6c00b-34b3-4a78-8136-af17e92293c0 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:12.730072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6c00b-34b3-4a78-8136-af17e92293c0 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:12.730344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6c00b-34b3-4a78-8136-af17e92293c0 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:12.731059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6c00b-34b3-4a78-8136-af17e92293c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:12.731521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6c00b-34b3-4a78-8136-af17e92293c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:12.732915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6c00b-34b3-4a78-8136-af17e92293c0 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:12.737857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 646/1291] 10.222.0.87 () {56 vars in 1116 bytes} [Fri Jun 17 12:42:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:12.744010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:12.851623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee58e5-ad1b-46b2-8d03-83d235157116 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:12.887908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be936-04ed-4efc-83e6-16cbbe7940cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:12.888693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be936-04ed-4efc-83e6-16cbbe7940cf None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:12.888982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be936-04ed-4efc-83e6-16cbbe7940cf None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:12.889265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be936-04ed-4efc-83e6-16cbbe7940cf None admin] PATH_INFO: `/v3/services` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:12.890011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be936-04ed-4efc-83e6-16cbbe7940cf None admin] RBAC: Adding query filter params (type=placement) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:12.890289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be936-04ed-4efc-83e6-16cbbe7940cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:12.891770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be936-04ed-4efc-83e6-16cbbe7940cf None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:12.896481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 646/1292] 10.222.0.87 () {56 vars in 1116 bytes} [Fri Jun 17 12:42:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:15.013557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9653-5c72-4d8a-9300-de42f1e23f16 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:15.014155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9653-5c72-4d8a-9300-de42f1e23f16 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:15.014155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9653-5c72-4d8a-9300-de42f1e23f16 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:15.015547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 647/1293] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:42:15.021839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19790-1c2f-4803-8763-23a2eb1056e3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:15.022553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19790-1c2f-4803-8763-23a2eb1056e3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:15.022701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19790-1c2f-4803-8763-23a2eb1056e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:15.043014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b8b19790-1c2f-4803-8763-23a2eb1056e3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:42:15.044447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b8b19790-1c2f-4803-8763-23a2eb1056e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:42:15.074756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:15.114985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 647/1294] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:42:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:42:15.136993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9653-5c72-4d8a-9300-de42f1e23f16 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:15.171284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:15.244286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9653-5c72-4d8a-9300-de42f1e23f16 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:15.280029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bb265-793b-4c17-b303-1a216552ae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:15.280621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb265-793b-4c17-b303-1a216552ae18 None admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:15.280911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb265-793b-4c17-b303-1a216552ae18 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:15.281182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb265-793b-4c17-b303-1a216552ae18 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:15.281914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bb265-793b-4c17-b303-1a216552ae18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:15.283299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bb265-793b-4c17-b303-1a216552ae18 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:15.299591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 648/1295] 10.222.0.87 () {60 vars in 1141 bytes} [Fri Jun 17 12:42:15 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:42:17.448653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466432f-4746-4c06-9c37-9a3f00eba27b None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:17.449499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466432f-4746-4c06-9c37-9a3f00eba27b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:17.449499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466432f-4746-4c06-9c37-9a3f00eba27b None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:17.451364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 648/1296] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:42:17.457693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2c14d-4ab9-4f3c-8592-07880b6c9708 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:17.458144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2c14d-4ab9-4f3c-8592-07880b6c9708 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:17.458239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2c14d-4ab9-4f3c-8592-07880b6c9708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:17.485389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d5b2c14d-4ab9-4f3c-8592-07880b6c9708 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:42:17.486573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d5b2c14d-4ab9-4f3c-8592-07880b6c9708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:42:17.572597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 649/1297] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:42:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:42:17.596873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466432f-4746-4c06-9c37-9a3f00eba27b None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:17.704764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466432f-4746-4c06-9c37-9a3f00eba27b None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:17.739094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:17.739919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:17.740202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:17.740473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:17.741127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:17.742521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:17.754467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:17.755745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:17.756825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 649/1298] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:42:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:17.764160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2c14d-4ab9-4f3c-8592-07880b6c9708 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:17.784438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:17.870358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2c14d-4ab9-4f3c-8592-07880b6c9708 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:17.905635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:17.906289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:17.906569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:17.906834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:17.907574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:17.907876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:17.909233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:17.914464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 650/1299] 10.222.0.87 () {56 vars in 1116 bytes} [Fri Jun 17 12:42:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:17.921841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:17.961553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:18.030482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23a5c2-8afa-4137-9fc4-8fa3a5a0e0bc None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:18.065064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dbf5e-1285-4a6c-a099-9d84caeb6b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:18.065715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dbf5e-1285-4a6c-a099-9d84caeb6b60 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:18.065994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dbf5e-1285-4a6c-a099-9d84caeb6b60 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:18.066265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dbf5e-1285-4a6c-a099-9d84caeb6b60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:18.067054 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dbf5e-1285-4a6c-a099-9d84caeb6b60 None admin] RBAC: Adding query filter params (interface=public, service_id=939ac6544b62480e85f84b23c3aeb978, region_id=RegionOne) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:18.067404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dbf5e-1285-4a6c-a099-9d84caeb6b60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:18.068831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dbf5e-1285-4a6c-a099-9d84caeb6b60 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:18.073952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 650/1300] 10.222.0.87 () {56 vars in 1251 bytes} [Fri Jun 17 12:42:17 2022] GET /identity/v3/endpoints?service_id=939ac6544b62480e85f84b23c3aeb978&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:18.080344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:18.172013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:18.191153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f216dc2-1469-49ea-975d-ce0bed6adad2 None admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:18.233164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ae504-700d-4c2e-a8fc-215f7c14710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:18.233758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae504-700d-4c2e-a8fc-215f7c14710d None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:18.234042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae504-700d-4c2e-a8fc-215f7c14710d None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:18.234317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae504-700d-4c2e-a8fc-215f7c14710d None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:18.235002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ae504-700d-4c2e-a8fc-215f7c14710d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:18.236424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ae504-700d-4c2e-a8fc-215f7c14710d None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:18.243579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 651/1301] 10.222.0.87 () {56 vars in 1087 bytes} [Fri Jun 17 12:42:18 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:42:20.356958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac526e2-5bc7-4116-8446-04cd2d89e1a6 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:20.357935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac526e2-5bc7-4116-8446-04cd2d89e1a6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:20.357935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac526e2-5bc7-4116-8446-04cd2d89e1a6 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:20.360135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 651/1302] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:42:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:42:20.369194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3046219-b4ee-419b-8b6a-bb8487ed61f3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:20.369900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3046219-b4ee-419b-8b6a-bb8487ed61f3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:20.370054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3046219-b4ee-419b-8b6a-bb8487ed61f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:20.389853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-e3046219-b4ee-419b-8b6a-bb8487ed61f3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:42:20.391042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e3046219-b4ee-419b-8b6a-bb8487ed61f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:42:20.462240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 652/1303] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:42:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 12:42:20.485800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac526e2-5bc7-4116-8446-04cd2d89e1a6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:20.587598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:20.596004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac526e2-5bc7-4116-8446-04cd2d89e1a6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:20.632704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:20.633430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:20.633714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:20.634010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:20.634658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:20.636073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 12:42:20.648471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 12:42:20.649626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 12:42:20.650493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 652/1304] 10.222.0.87 () {56 vars in 1117 bytes} [Fri Jun 17 12:42:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 12:42:20.656499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3046219-b4ee-419b-8b6a-bb8487ed61f3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:20.740476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:20.767109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3046219-b4ee-419b-8b6a-bb8487ed61f3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:20.801783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7968c2-69d4-4b58-af5f-684a13885808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:20.802442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7968c2-69d4-4b58-af5f-684a13885808 None admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:20.802728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7968c2-69d4-4b58-af5f-684a13885808 None admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:20.803330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7968c2-69d4-4b58-af5f-684a13885808 None admin] PATH_INFO: `/v3/services` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:20.804103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7968c2-69d4-4b58-af5f-684a13885808 None admin] RBAC: Adding query filter params (name=placement) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:42:20.804411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7968c2-69d4-4b58-af5f-684a13885808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:20.805772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7968c2-69d4-4b58-af5f-684a13885808 None admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:20.811248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 653/1305] 10.222.0.87 () {56 vars in 1116 bytes} [Fri Jun 17 12:42:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:42:20.817369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:42:20.922252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411664a-f04d-4cdb-9eb3-3891a868ccd3 None admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:42:20.957514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d873f20-0cac-4e6b-b100-e9405796ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:42:20.958169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d873f20-0cac-4e6b-b100-e9405796ca17 None admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:42:20.958455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d873f20-0cac-4e6b-b100-e9405796ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:42:20.958728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d873f20-0cac-4e6b-b100-e9405796ca17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:42:20.959515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d873f20-0cac-4e6b-b100-e9405796ca17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:42:20.960886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d873f20-0cac-4e6b-b100-e9405796ca17 None admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:42:20.983828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 653/1306] 10.222.0.87 () {60 vars in 1144 bytes} [Fri Jun 17 12:42:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:42:30.808389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=30 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:42:30.978920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:24.587799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523a101-53c4-48e0-bee8-47f5b898ff2f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:24.588409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523a101-53c4-48e0-bee8-47f5b898ff2f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:24.588409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523a101-53c4-48e0-bee8-47f5b898ff2f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:24.589960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 654/1307] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:24.595893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f691-63c9-4d9c-8cfe-7e159ff59bce None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:24.596445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f691-63c9-4d9c-8cfe-7e159ff59bce None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:24.596445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f691-63c9-4d9c-8cfe-7e159ff59bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:24.706570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-aa29f691-63c9-4d9c-8cfe-7e159ff59bce None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:24.707838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-aa29f691-63c9-4d9c-8cfe-7e159ff59bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:24.840928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 654/1308] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:24.918420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897353b-5266-4db5-b1d3-7de651e8055c None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:24.918965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897353b-5266-4db5-b1d3-7de651e8055c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:24.919359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897353b-5266-4db5-b1d3-7de651e8055c None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:24.920955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 655/1309] 10.222.0.87 () {54 vars in 964 bytes} [Fri Jun 17 12:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:24.935769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49376-410b-45ae-9a75-313ae2814e9c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:24.937524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49376-410b-45ae-9a75-313ae2814e9c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:24.938714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49376-410b-45ae-9a75-313ae2814e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:25.174171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-eee49376-410b-45ae-9a75-313ae2814e9c None None] MFA Rules not processed for user `40f6ae0e5e164461b4c51b829be9a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:25.175490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-eee49376-410b-45ae-9a75-313ae2814e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:25.196963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:25.281782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 655/1310] 10.222.0.87 () {58 vars in 1061 bytes} [Fri Jun 17 12:43:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:25.291328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897353b-5266-4db5-b1d3-7de651e8055c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:25.293197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6897353b-5266-4db5-b1d3-7de651e8055c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:25.444483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897353b-5266-4db5-b1d3-7de651e8055c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:25.536115 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f578f-1c39-42f9-b6cc-d9a534c33caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:25.536821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f578f-1c39-42f9-b6cc-d9a534c33caa service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:25.537113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f578f-1c39-42f9-b6cc-d9a534c33caa service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:25.537387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f578f-1c39-42f9-b6cc-d9a534c33caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:25.539114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c87f578f-1c39-42f9-b6cc-d9a534c33caa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:25.612681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:25.676308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f578f-1c39-42f9-b6cc-d9a534c33caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:25.678078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f578f-1c39-42f9-b6cc-d9a534c33caa service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:25.701564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 656/1311] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:28.162112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2023f-0578-47a6-bdaf-c2fac4efd13f None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:28.162686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2023f-0578-47a6-bdaf-c2fac4efd13f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:28.162686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2023f-0578-47a6-bdaf-c2fac4efd13f None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:28.164052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 656/1312] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:28.169430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0eec0-ee18-4116-969e-fd8a3c7ede39 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:28.169846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0eec0-ee18-4116-969e-fd8a3c7ede39 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:28.169944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0eec0-ee18-4116-969e-fd8a3c7ede39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:28.251039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9ef0eec0-ee18-4116-969e-fd8a3c7ede39 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:28.252277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9ef0eec0-ee18-4116-969e-fd8a3c7ede39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:28.352565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 657/1313] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:28.404723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581a8ac-8e8c-4406-b5db-1ac33d0e3c0c None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:28.405102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581a8ac-8e8c-4406-b5db-1ac33d0e3c0c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:28.405447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581a8ac-8e8c-4406-b5db-1ac33d0e3c0c None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:28.406905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 657/1314] 10.222.0.87 () {54 vars in 964 bytes} [Fri Jun 17 12:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:28.412928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3152-a538-4de1-938a-34dfde12a3fb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:28.413222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3152-a538-4de1-938a-34dfde12a3fb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:28.413500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3152-a538-4de1-938a-34dfde12a3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:28.425368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:28.495125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-87fb3152-a538-4de1-938a-34dfde12a3fb None None] MFA Rules not processed for user `40f6ae0e5e164461b4c51b829be9a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:28.496386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-87fb3152-a538-4de1-938a-34dfde12a3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:28.574323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 658/1315] 10.222.0.87 () {58 vars in 1061 bytes} [Fri Jun 17 12:43:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:28.583628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581a8ac-8e8c-4406-b5db-1ac33d0e3c0c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:28.585189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4581a8ac-8e8c-4406-b5db-1ac33d0e3c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:28.703103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581a8ac-8e8c-4406-b5db-1ac33d0e3c0c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:28.780442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:28.783114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d15323-c48c-48bd-b146-e92067cf4268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:28.783847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d15323-c48c-48bd-b146-e92067cf4268 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:28.784137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d15323-c48c-48bd-b146-e92067cf4268 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:28.784487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d15323-c48c-48bd-b146-e92067cf4268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:28.787617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b8d15323-c48c-48bd-b146-e92067cf4268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:28.900551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d15323-c48c-48bd-b146-e92067cf4268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:28.901974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d15323-c48c-48bd-b146-e92067cf4268 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:28.920833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 658/1316] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:31.571175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56088f-9d5f-4755-bc2b-d9b33a13e5bc None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:31.571856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56088f-9d5f-4755-bc2b-d9b33a13e5bc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:31.571856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56088f-9d5f-4755-bc2b-d9b33a13e5bc None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:31.573146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 659/1317] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:31.578593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7095-adab-48e0-a1f8-3e6138c4d070 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:31.579059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7095-adab-48e0-a1f8-3e6138c4d070 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:31.579153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7095-adab-48e0-a1f8-3e6138c4d070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:31.651349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d93a7095-adab-48e0-a1f8-3e6138c4d070 None None] MFA Rules not processed for user `1e936b4c92074da5a0a5d5ff77dda5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:31.652642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d93a7095-adab-48e0-a1f8-3e6138c4d070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:31.707696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:31.753712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 659/1318] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:31.808229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56088f-9d5f-4755-bc2b-d9b33a13e5bc None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:31.809784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4c56088f-9d5f-4755-bc2b-d9b33a13e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:31.918852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56088f-9d5f-4755-bc2b-d9b33a13e5bc None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:31.921774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:31.994735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d4b7d-f8d8-4b7c-b72a-f1cede756ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:31.995413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d4b7d-f8d8-4b7c-b72a-f1cede756ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:31.995707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d4b7d-f8d8-4b7c-b72a-f1cede756ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:31.995979 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d4b7d-f8d8-4b7c-b72a-f1cede756ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:31.997649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ee4d4b7d-f8d8-4b7c-b72a-f1cede756ff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:32.116196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d4b7d-f8d8-4b7c-b72a-f1cede756ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:32.117639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d4b7d-f8d8-4b7c-b72a-f1cede756ff5 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:32.136820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 660/1319] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:34.983977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db214c7-5dff-468e-9cb2-341b43fc6b12 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:34.984861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db214c7-5dff-468e-9cb2-341b43fc6b12 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:34.984861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db214c7-5dff-468e-9cb2-341b43fc6b12 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:34.986983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 660/1320] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:34.993508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c828542-b01a-4246-91b2-1712b4f768d0 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:34.994214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c828542-b01a-4246-91b2-1712b4f768d0 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:34.994414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c828542-b01a-4246-91b2-1712b4f768d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:35.044299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:35.078354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7c828542-b01a-4246-91b2-1712b4f768d0 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:35.079901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7c828542-b01a-4246-91b2-1712b4f768d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:35.187681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 661/1321] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:35.234172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db214c7-5dff-468e-9cb2-341b43fc6b12 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:35.235681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7db214c7-5dff-468e-9cb2-341b43fc6b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:35.349549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db214c7-5dff-468e-9cb2-341b43fc6b12 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:35.434796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70b2a3-8c43-43a7-8e1d-c27e49eb6d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:35.435475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b2a3-8c43-43a7-8e1d-c27e49eb6d43 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:35.435766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b2a3-8c43-43a7-8e1d-c27e49eb6d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:35.436034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b2a3-8c43-43a7-8e1d-c27e49eb6d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:35.437690 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4e70b2a3-8c43-43a7-8e1d-c27e49eb6d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:35.445491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:35.559192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70b2a3-8c43-43a7-8e1d-c27e49eb6d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:35.560951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70b2a3-8c43-43a7-8e1d-c27e49eb6d43 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:35.579514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 661/1322] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:38.377751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5cd6-90e8-4696-a018-65650166f348 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:38.378281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5cd6-90e8-4696-a018-65650166f348 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:38.378281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5cd6-90e8-4696-a018-65650166f348 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:38.379960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 662/1323] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:38.385445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c401c8a-ed3a-4f9d-8fd1-99c0dde3f289 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:38.385877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c401c8a-ed3a-4f9d-8fd1-99c0dde3f289 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:38.386023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c401c8a-ed3a-4f9d-8fd1-99c0dde3f289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:38.458173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3c401c8a-ed3a-4f9d-8fd1-99c0dde3f289 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:38.459671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3c401c8a-ed3a-4f9d-8fd1-99c0dde3f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:38.527518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:38.574176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 662/1324] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:38.612021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd5cd6-90e8-4696-a018-65650166f348 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:38.614511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4ebd5cd6-90e8-4696-a018-65650166f348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:38.737281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd5cd6-90e8-4696-a018-65650166f348 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:38.754656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:38.826168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aafc2-6535-47c7-8c0d-66bd9a3aef08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:38.826845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aafc2-6535-47c7-8c0d-66bd9a3aef08 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:38.827132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aafc2-6535-47c7-8c0d-66bd9a3aef08 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:38.827432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aafc2-6535-47c7-8c0d-66bd9a3aef08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:38.829086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-392aafc2-6535-47c7-8c0d-66bd9a3aef08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:38.948294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aafc2-6535-47c7-8c0d-66bd9a3aef08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:38.949683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aafc2-6535-47c7-8c0d-66bd9a3aef08 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:38.968406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 663/1325] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:41.173558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fda400-feba-40c6-8c0b-270e38688006 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:41.174106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fda400-feba-40c6-8c0b-270e38688006 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:41.174106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fda400-feba-40c6-8c0b-270e38688006 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:41.175540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 663/1326] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:41.181313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7eddb-10d4-42bc-ad91-eb35d9cb649e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:41.181716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7eddb-10d4-42bc-ad91-eb35d9cb649e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:41.181869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7eddb-10d4-42bc-ad91-eb35d9cb649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:41.232043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:41.251166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0ff7eddb-10d4-42bc-ad91-eb35d9cb649e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:41.252388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0ff7eddb-10d4-42bc-ad91-eb35d9cb649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:41.353153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 664/1327] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:41.388899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fda400-feba-40c6-8c0b-270e38688006 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:41.390564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-46fda400-feba-40c6-8c0b-270e38688006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:41.506248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fda400-feba-40c6-8c0b-270e38688006 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:41.582992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435f77c-fb52-4b1e-b8db-11f7aff375ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:41.583815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435f77c-fb52-4b1e-b8db-11f7aff375ca service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:41.584193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435f77c-fb52-4b1e-b8db-11f7aff375ca service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:41.584541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435f77c-fb52-4b1e-b8db-11f7aff375ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:41.586310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8435f77c-fb52-4b1e-b8db-11f7aff375ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:41.608558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:41.703731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435f77c-fb52-4b1e-b8db-11f7aff375ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:41.705182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435f77c-fb52-4b1e-b8db-11f7aff375ca service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:41.723817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 664/1328] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:43.905905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0386e8-e0cd-43fa-a111-a3eb5f201f22 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:43.906464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0386e8-e0cd-43fa-a111-a3eb5f201f22 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:43.906698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0386e8-e0cd-43fa-a111-a3eb5f201f22 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:43.908180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 665/1329] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:43.914143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca48c4a-849d-4263-9125-3ab1fe527c47 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:43.914839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca48c4a-849d-4263-9125-3ab1fe527c47 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:43.914839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca48c4a-849d-4263-9125-3ab1fe527c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:43.986408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-7ca48c4a-849d-4263-9125-3ab1fe527c47 None None] MFA Rules not processed for user `1e936b4c92074da5a0a5d5ff77dda5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:43.987599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7ca48c4a-849d-4263-9125-3ab1fe527c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:44.059407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:44.092543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 665/1330] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:44.130849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0386e8-e0cd-43fa-a111-a3eb5f201f22 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:44.132591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4e0386e8-e0cd-43fa-a111-a3eb5f201f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:44.271657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0386e8-e0cd-43fa-a111-a3eb5f201f22 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:44.292416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:44.349632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c59ff-5816-48ab-ba51-bfea972eb39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:44.350333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c59ff-5816-48ab-ba51-bfea972eb39f service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:44.350617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c59ff-5816-48ab-ba51-bfea972eb39f service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:44.350889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c59ff-5816-48ab-ba51-bfea972eb39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:44.353230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-245c59ff-5816-48ab-ba51-bfea972eb39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:44.472181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c59ff-5816-48ab-ba51-bfea972eb39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:44.473836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c59ff-5816-48ab-ba51-bfea972eb39f service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:44.494548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 666/1331] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:46.891335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca894-e955-4c67-b0a8-c3d447a6be42 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:46.892248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca894-e955-4c67-b0a8-c3d447a6be42 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:46.892248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ca894-e955-4c67-b0a8-c3d447a6be42 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:46.894374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 666/1332] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:46.903045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282b860-c7c9-4ea7-9134-3780f63ede55 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:46.903498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282b860-c7c9-4ea7-9134-3780f63ede55 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:46.903673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282b860-c7c9-4ea7-9134-3780f63ede55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:46.970911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:46.974391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b282b860-c7c9-4ea7-9134-3780f63ede55 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:46.976152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b282b860-c7c9-4ea7-9134-3780f63ede55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:47.077227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 667/1333] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:47.115440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ca894-e955-4c67-b0a8-c3d447a6be42 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:47.116677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-de8ca894-e955-4c67-b0a8-c3d447a6be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:47.224366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ca894-e955-4c67-b0a8-c3d447a6be42 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:47.300279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236c81e-a514-43aa-a6b3-d5e0781a58eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:47.300931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236c81e-a514-43aa-a6b3-d5e0781a58eb service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:47.301259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236c81e-a514-43aa-a6b3-d5e0781a58eb service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:47.301539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236c81e-a514-43aa-a6b3-d5e0781a58eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:47.303193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3236c81e-a514-43aa-a6b3-d5e0781a58eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:47.330165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:47.429884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236c81e-a514-43aa-a6b3-d5e0781a58eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:47.431291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236c81e-a514-43aa-a6b3-d5e0781a58eb service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:47.449994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 667/1334] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:49.620675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bf66c-4c7c-4616-bad4-ac93c799419f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:49.621223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bf66c-4c7c-4616-bad4-ac93c799419f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:49.621354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bf66c-4c7c-4616-bad4-ac93c799419f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:49.623455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 668/1335] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:49.632570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33c5e7-75df-4c23-8a21-e929410d68c3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:49.632981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33c5e7-75df-4c23-8a21-e929410d68c3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:49.633224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33c5e7-75df-4c23-8a21-e929410d68c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:49.713126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-5d33c5e7-75df-4c23-8a21-e929410d68c3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:49.714329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5d33c5e7-75df-4c23-8a21-e929410d68c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:49.790494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:49.824882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 668/1336] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:49.874733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bf66c-4c7c-4616-bad4-ac93c799419f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:49.877032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-072bf66c-4c7c-4616-bad4-ac93c799419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:50.004309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bf66c-4c7c-4616-bad4-ac93c799419f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:50.043040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:50.094848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe7ce1-e191-4370-a526-6c10f07a17c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:50.095531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe7ce1-e191-4370-a526-6c10f07a17c6 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:50.095817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe7ce1-e191-4370-a526-6c10f07a17c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:50.096096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe7ce1-e191-4370-a526-6c10f07a17c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:50.097775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2ebe7ce1-e191-4370-a526-6c10f07a17c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:50.211593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe7ce1-e191-4370-a526-6c10f07a17c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:50.212963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe7ce1-e191-4370-a526-6c10f07a17c6 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:50.231976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 669/1337] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:52.673295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b4e87-ca38-4101-a903-e05970fa29a3 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:52.674828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b4e87-ca38-4101-a903-e05970fa29a3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:52.675293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b4e87-ca38-4101-a903-e05970fa29a3 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:52.677328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 669/1338] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:52.682818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabc1a-a1db-496f-9ef3-c6191d47d500 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:52.683275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabc1a-a1db-496f-9ef3-c6191d47d500 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:52.683400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabc1a-a1db-496f-9ef3-c6191d47d500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:52.751939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a4fabc1a-a1db-496f-9ef3-c6191d47d500 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:52.753139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a4fabc1a-a1db-496f-9ef3-c6191d47d500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:52.761530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:52.854780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 670/1339] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:52.894387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b4e87-ca38-4101-a903-e05970fa29a3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:52.897293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8c0b4e87-ca38-4101-a903-e05970fa29a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:53.032289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b4e87-ca38-4101-a903-e05970fa29a3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:53.107334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d79a3-d033-4343-b095-2b4819a01442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:53.108230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d79a3-d033-4343-b095-2b4819a01442 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:53.108230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d79a3-d033-4343-b095-2b4819a01442 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:53.108516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d79a3-d033-4343-b095-2b4819a01442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:53.110212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d55d79a3-d033-4343-b095-2b4819a01442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:53.147834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:53.226099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d79a3-d033-4343-b095-2b4819a01442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:53.227617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d79a3-d033-4343-b095-2b4819a01442 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:53.245786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 670/1340] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:43:57.379410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f592-5a4d-49ba-8aa9-73a390abd188 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:57.380143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f592-5a4d-49ba-8aa9-73a390abd188 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:57.380504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f592-5a4d-49ba-8aa9-73a390abd188 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:57.382798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 671/1341] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:43:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:43:57.388350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0ceac-b965-4263-b749-be4f17041b57 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:57.388920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0ceac-b965-4263-b749-be4f17041b57 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:57.388920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0ceac-b965-4263-b749-be4f17041b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:57.467207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-6ef0ceac-b965-4263-b749-be4f17041b57 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:43:57.468369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6ef0ceac-b965-4263-b749-be4f17041b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:57.568011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:43:57.581984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 671/1342] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:43:57.621600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70f592-5a4d-49ba-8aa9-73a390abd188 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:43:57.625088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8c70f592-5a4d-49ba-8aa9-73a390abd188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:43:57.742756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70f592-5a4d-49ba-8aa9-73a390abd188 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:43:57.746703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031c831-fbb1-49ae-902a-6274cd0e0a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:43:57.747361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c831-fbb1-49ae-902a-6274cd0e0a65 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:43:57.747674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c831-fbb1-49ae-902a-6274cd0e0a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:43:57.747951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c831-fbb1-49ae-902a-6274cd0e0a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:43:57.757648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031c831-fbb1-49ae-902a-6274cd0e0a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:43:57.759096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031c831-fbb1-49ae-902a-6274cd0e0a65 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:43:57.768938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 672/1343] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:00.707740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2d52-e245-427f-b865-ae1cb59c0bde None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:00.708375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2d52-e245-427f-b865-ae1cb59c0bde None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:00.708375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2d52-e245-427f-b865-ae1cb59c0bde None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:00.709895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 672/1344] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:00.715535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919c66-15a2-4c8c-bb87-5bef5a9ce42b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:00.716003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919c66-15a2-4c8c-bb87-5bef5a9ce42b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:00.716111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919c66-15a2-4c8c-bb87-5bef5a9ce42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:00.745074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f7919c66-15a2-4c8c-bb87-5bef5a9ce42b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:00.746293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f7919c66-15a2-4c8c-bb87-5bef5a9ce42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:00.775831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:44:00.828492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 673/1345] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:00.871885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2d52-e245-427f-b865-ae1cb59c0bde None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:00.882984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2d52-e245-427f-b865-ae1cb59c0bde None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:00.886525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43162d-3088-4d8d-ace5-28d7317bcb0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:00.887171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43162d-3088-4d8d-ace5-28d7317bcb0d service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:00.887490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43162d-3088-4d8d-ace5-28d7317bcb0d service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:00.887767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43162d-3088-4d8d-ace5-28d7317bcb0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:00.897126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43162d-3088-4d8d-ace5-28d7317bcb0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:00.898566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43162d-3088-4d8d-ace5-28d7317bcb0d service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:00.908136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 673/1346] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:05.249413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171e206-d0dd-49c3-92bb-7fc9002862b8 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:05.250086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171e206-d0dd-49c3-92bb-7fc9002862b8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:05.250086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171e206-d0dd-49c3-92bb-7fc9002862b8 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:05.258606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e24d50-b5b4-4f56-9a64-cef64bce1a30 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:05.259114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e24d50-b5b4-4f56-9a64-cef64bce1a30 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:05.259209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e24d50-b5b4-4f56-9a64-cef64bce1a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:05.261072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 674/1347] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:05.279791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-27e24d50-b5b4-4f56-9a64-cef64bce1a30 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:05.281072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-27e24d50-b5b4-4f56-9a64-cef64bce1a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:05.292821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 674/1348] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:05.331028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171e206-d0dd-49c3-92bb-7fc9002862b8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:05.343926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171e206-d0dd-49c3-92bb-7fc9002862b8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:05.348885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461da16f-83ec-4375-bea1-4559dd3d70c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:05.349871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461da16f-83ec-4375-bea1-4559dd3d70c7 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:05.349871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461da16f-83ec-4375-bea1-4559dd3d70c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:05.350114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461da16f-83ec-4375-bea1-4559dd3d70c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:05.361399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461da16f-83ec-4375-bea1-4559dd3d70c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:05.362865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461da16f-83ec-4375-bea1-4559dd3d70c7 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:05.374170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 675/1349] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:10.385777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2516f-e368-4c09-8695-c75178e055d8 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:10.386228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2516f-e368-4c09-8695-c75178e055d8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:10.386373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2516f-e368-4c09-8695-c75178e055d8 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:10.403421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 675/1350] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:44:10 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:10.426548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60aec-1f43-49a2-b0d2-2df193a7fe8c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:10.427050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60aec-1f43-49a2-b0d2-2df193a7fe8c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:10.427151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60aec-1f43-49a2-b0d2-2df193a7fe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:10.552846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a5e60aec-1f43-49a2-b0d2-2df193a7fe8c None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:10.554843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a5e60aec-1f43-49a2-b0d2-2df193a7fe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:10.638844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 676/1351] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:44:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:14.711320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e642a9-610d-4d62-bfa0-489cab94e1a4 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:14.711924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e642a9-610d-4d62-bfa0-489cab94e1a4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:14.711924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e642a9-610d-4d62-bfa0-489cab94e1a4 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:14.713453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 676/1352] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:14.718466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8b212-6cab-4f1c-b203-988c5da1b13c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:14.718906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8b212-6cab-4f1c-b203-988c5da1b13c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:14.719231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8b212-6cab-4f1c-b203-988c5da1b13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:14.738947 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-95b8b212-6cab-4f1c-b203-988c5da1b13c None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:14.740348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-95b8b212-6cab-4f1c-b203-988c5da1b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:14.750406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 677/1353] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:15.274442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=13 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:44:16.498097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46615bd-6afa-41e8-9df5-6076e703fe03 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:16.498762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46615bd-6afa-41e8-9df5-6076e703fe03 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:16.498762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46615bd-6afa-41e8-9df5-6076e703fe03 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:16.500311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 677/1354] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:16.505652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1c47c-f06f-481b-8bd1-acc6edb4cdf1 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:16.506103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1c47c-f06f-481b-8bd1-acc6edb4cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:16.506336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1c47c-f06f-481b-8bd1-acc6edb4cdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:16.526183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-60b1c47c-f06f-481b-8bd1-acc6edb4cdf1 None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:16.527408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-60b1c47c-f06f-481b-8bd1-acc6edb4cdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:16.537350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 678/1355] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:44:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:20.261345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4fa56-3daa-4b4a-921a-bd159cd6e346 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:20.261920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4fa56-3daa-4b4a-921a-bd159cd6e346 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:20.261920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4fa56-3daa-4b4a-921a-bd159cd6e346 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:20.263449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 678/1356] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:20.269158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d550b2-6444-4be6-a7bb-df9c412435d8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:20.269653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d550b2-6444-4be6-a7bb-df9c412435d8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:20.269844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d550b2-6444-4be6-a7bb-df9c412435d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:20.289653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a3d550b2-6444-4be6-a7bb-df9c412435d8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:20.290996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a3d550b2-6444-4be6-a7bb-df9c412435d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:20.301192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 679/1357] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:20.360739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdb8b5-aa64-4028-8620-a5b51b9f6413 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:20.361010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdb8b5-aa64-4028-8620-a5b51b9f6413 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:20.361474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdb8b5-aa64-4028-8620-a5b51b9f6413 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:20.362994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 679/1358] 10.222.0.87 () {54 vars in 961 bytes} [Fri Jun 17 12:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:20.373720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143be1-76f3-4482-8f2d-c1c5e804f182 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:20.374359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143be1-76f3-4482-8f2d-c1c5e804f182 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:20.374803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143be1-76f3-4482-8f2d-c1c5e804f182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:20.422196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:44:20.431321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f1143be1-76f3-4482-8f2d-c1c5e804f182 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:20.432548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f1143be1-76f3-4482-8f2d-c1c5e804f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:20.527524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 680/1359] 10.222.0.87 () {58 vars in 1058 bytes} [Fri Jun 17 12:44:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:20.536044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdb8b5-aa64-4028-8620-a5b51b9f6413 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:20.552406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdb8b5-aa64-4028-8620-a5b51b9f6413 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:20.558526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c3611-333c-4d0d-a554-229c68d5d6e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:20.559472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c3611-333c-4d0d-a554-229c68d5d6e5 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:20.559906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c3611-333c-4d0d-a554-229c68d5d6e5 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:20.560353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c3611-333c-4d0d-a554-229c68d5d6e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:20.570111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c3611-333c-4d0d-a554-229c68d5d6e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:20.573417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c3611-333c-4d0d-a554-229c68d5d6e5 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:20.584288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 680/1360] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:22.910169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d92d8f-5b51-4101-af08-1437bb0e7795 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:22.910796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d92d8f-5b51-4101-af08-1437bb0e7795 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:22.910796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d92d8f-5b51-4101-af08-1437bb0e7795 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:22.912329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 681/1361] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:22.918352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d113f-1aea-4208-9c34-ae86424ced8f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:22.918914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d113f-1aea-4208-9c34-ae86424ced8f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:22.919045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d113f-1aea-4208-9c34-ae86424ced8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:22.939002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e83d113f-1aea-4208-9c34-ae86424ced8f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:22.940320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e83d113f-1aea-4208-9c34-ae86424ced8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:22.949414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 681/1362] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:22.995335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d92d8f-5b51-4101-af08-1437bb0e7795 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:23.010621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d92d8f-5b51-4101-af08-1437bb0e7795 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:23.017904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ba7c0-2bcf-4409-96ea-68e4b75a7f1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:23.018824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ba7c0-2bcf-4409-96ea-68e4b75a7f1a service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:23.019309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ba7c0-2bcf-4409-96ea-68e4b75a7f1a service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:23.019842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ba7c0-2bcf-4409-96ea-68e4b75a7f1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:23.029336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ba7c0-2bcf-4409-96ea-68e4b75a7f1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:23.031224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ba7c0-2bcf-4409-96ea-68e4b75a7f1a service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:23.041928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 682/1363] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:25.502050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0d18e-6e27-46ac-894e-49d91c55b441 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:25.502546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0d18e-6e27-46ac-894e-49d91c55b441 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:25.502659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0d18e-6e27-46ac-894e-49d91c55b441 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:25.504084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 682/1364] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:25.509543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7712e64-32d0-4152-9bdc-44a406cf3b07 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:25.509967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7712e64-32d0-4152-9bdc-44a406cf3b07 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:25.510142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7712e64-32d0-4152-9bdc-44a406cf3b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:25.536334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a7712e64-32d0-4152-9bdc-44a406cf3b07 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:25.537542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a7712e64-32d0-4152-9bdc-44a406cf3b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:25.546558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 683/1365] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:25.592646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25ee95-4a5f-4056-ac92-85af46bb83fb None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:25.592908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25ee95-4a5f-4056-ac92-85af46bb83fb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:25.593214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25ee95-4a5f-4056-ac92-85af46bb83fb None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:25.594575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 683/1366] 10.222.0.87 () {54 vars in 961 bytes} [Fri Jun 17 12:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:25.599976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75eb3d-7a5a-42f2-b81e-2a5d2c302bb7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:25.600257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75eb3d-7a5a-42f2-b81e-2a5d2c302bb7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:25.600529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75eb3d-7a5a-42f2-b81e-2a5d2c302bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:25.621825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8e75eb3d-7a5a-42f2-b81e-2a5d2c302bb7 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:25.623022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8e75eb3d-7a5a-42f2-b81e-2a5d2c302bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:25.633152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 684/1367] 10.222.0.87 () {58 vars in 1058 bytes} [Fri Jun 17 12:44:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:25.641375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25ee95-4a5f-4056-ac92-85af46bb83fb None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:25.659966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25ee95-4a5f-4056-ac92-85af46bb83fb None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:25.666585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143d4d5-0c16-483e-8e38-eb23bb6fb58b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:25.667244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143d4d5-0c16-483e-8e38-eb23bb6fb58b service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:25.667562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143d4d5-0c16-483e-8e38-eb23bb6fb58b service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:25.667840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143d4d5-0c16-483e-8e38-eb23bb6fb58b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:25.676699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143d4d5-0c16-483e-8e38-eb23bb6fb58b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:25.678307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143d4d5-0c16-483e-8e38-eb23bb6fb58b service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:25.688333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 684/1368] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:27.889885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7458a4-97a5-48cb-b514-92d0579ddc19 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:27.890510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7458a4-97a5-48cb-b514-92d0579ddc19 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:27.890510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7458a4-97a5-48cb-b514-92d0579ddc19 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:27.892105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 685/1369] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:27.897167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e89db-b5b7-40f4-b40d-6bf2f101d7d8 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:27.897609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e89db-b5b7-40f4-b40d-6bf2f101d7d8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:27.897708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e89db-b5b7-40f4-b40d-6bf2f101d7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:27.918253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-116e89db-b5b7-40f4-b40d-6bf2f101d7d8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:27.919470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-116e89db-b5b7-40f4-b40d-6bf2f101d7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:27.931459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 685/1370] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:27.972504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7458a4-97a5-48cb-b514-92d0579ddc19 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:27.987441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7458a4-97a5-48cb-b514-92d0579ddc19 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:27.993058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8dca1-848b-4b10-9cb8-9a2bf992a177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:27.993757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8dca1-848b-4b10-9cb8-9a2bf992a177 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:27.994032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8dca1-848b-4b10-9cb8-9a2bf992a177 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:27.994309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8dca1-848b-4b10-9cb8-9a2bf992a177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:28.003836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8dca1-848b-4b10-9cb8-9a2bf992a177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:28.005423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8dca1-848b-4b10-9cb8-9a2bf992a177 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:28.016195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 686/1371] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:28.376436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4957-c67b-482f-b39d-d7e2e2395de1 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:28.376895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4957-c67b-482f-b39d-d7e2e2395de1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:28.377278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4957-c67b-482f-b39d-d7e2e2395de1 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:28.378982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 686/1372] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:28.384725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e4bc-4292-4823-aad9-46834b7d19d1 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:28.385113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e4bc-4292-4823-aad9-46834b7d19d1 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:28.385393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e4bc-4292-4823-aad9-46834b7d19d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:28.412554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4a88e4bc-4292-4823-aad9-46834b7d19d1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:28.413739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4a88e4bc-4292-4823-aad9-46834b7d19d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:28.424141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 687/1373] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:28.736923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6bc95-fe99-46b9-9e96-68c867ff8c43 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:28.737332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6bc95-fe99-46b9-9e96-68c867ff8c43 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:28.738238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6bc95-fe99-46b9-9e96-68c867ff8c43 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:28.740018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 687/1374] 10.222.0.87 () {54 vars in 963 bytes} [Fri Jun 17 12:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:28.747168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376815-80f4-4edf-8698-5a18d0609f22 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:28.747465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376815-80f4-4edf-8698-5a18d0609f22 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:28.747770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376815-80f4-4edf-8698-5a18d0609f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:28.805253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-5f376815-80f4-4edf-8698-5a18d0609f22 None None] MFA Rules not processed for user `1315c22b7db84afd9ef41bf853b9819b`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:28.806463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5f376815-80f4-4edf-8698-5a18d0609f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:28.875202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 688/1375] 10.222.0.87 () {58 vars in 1060 bytes} [Fri Jun 17 12:44:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:28.882823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6bc95-fe99-46b9-9e96-68c867ff8c43 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:28.894330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6bc95-fe99-46b9-9e96-68c867ff8c43 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:28.898375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6a6a8-664f-4111-baa2-c20d5fa00841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:28.899717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6a6a8-664f-4111-baa2-c20d5fa00841 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:28.899970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6a6a8-664f-4111-baa2-c20d5fa00841 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:28.900274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6a6a8-664f-4111-baa2-c20d5fa00841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:28.909771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6a6a8-664f-4111-baa2-c20d5fa00841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:28.911275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6a6a8-664f-4111-baa2-c20d5fa00841 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:28.921368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 688/1376] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:30.207519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ebb3-8bc4-41e1-8b2a-5459719d8827 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:30.208111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ebb3-8bc4-41e1-8b2a-5459719d8827 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:30.208111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ebb3-8bc4-41e1-8b2a-5459719d8827 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:30.209509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 689/1377] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:30.215403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6649-0f1f-4893-a3a0-e7dc0fd31d72 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:30.215854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6649-0f1f-4893-a3a0-e7dc0fd31d72 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:30.215950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6649-0f1f-4893-a3a0-e7dc0fd31d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:30.233960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-12dd6649-0f1f-4893-a3a0-e7dc0fd31d72 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:30.235167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-12dd6649-0f1f-4893-a3a0-e7dc0fd31d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:30.244245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 689/1378] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:30.287795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ebb3-8bc4-41e1-8b2a-5459719d8827 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:30.302871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ebb3-8bc4-41e1-8b2a-5459719d8827 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:30.308820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57771e00-2841-4bd0-be6b-638faf580d9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:30.309466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57771e00-2841-4bd0-be6b-638faf580d9d service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:30.309748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57771e00-2841-4bd0-be6b-638faf580d9d service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:30.310018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57771e00-2841-4bd0-be6b-638faf580d9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:30.318822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57771e00-2841-4bd0-be6b-638faf580d9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:30.320500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57771e00-2841-4bd0-be6b-638faf580d9d service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:30.324538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:44:30.331704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 690/1379] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:32.550325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cabf-0545-48a3-8970-313cbac42a9a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:32.551311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cabf-0545-48a3-8970-313cbac42a9a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:32.551651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cabf-0545-48a3-8970-313cbac42a9a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:32.553161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 690/1380] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:32.558965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cdc51-3927-4701-8f4b-67769d39b0a5 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:32.559484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cdc51-3927-4701-8f4b-67769d39b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:32.559582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cdc51-3927-4701-8f4b-67769d39b0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:32.580493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-dc9cdc51-3927-4701-8f4b-67769d39b0a5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:32.581700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-dc9cdc51-3927-4701-8f4b-67769d39b0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:32.590624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 691/1381] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:32.636745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2cabf-0545-48a3-8970-313cbac42a9a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:32.652410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2cabf-0545-48a3-8970-313cbac42a9a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:32.657765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd19529-0db0-4802-b8ea-f7f6a2071d90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:32.658489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd19529-0db0-4802-b8ea-f7f6a2071d90 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:32.658836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd19529-0db0-4802-b8ea-f7f6a2071d90 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:32.659180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd19529-0db0-4802-b8ea-f7f6a2071d90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:32.668597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd19529-0db0-4802-b8ea-f7f6a2071d90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:32.670489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd19529-0db0-4802-b8ea-f7f6a2071d90 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:32.680997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 691/1382] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:34.847458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ae817-a501-446a-949a-45e16fdf1b39 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:34.847996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ae817-a501-446a-949a-45e16fdf1b39 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:34.847996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ae817-a501-446a-949a-45e16fdf1b39 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:34.849397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 692/1383] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:34.854806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c53d29-1d9a-4539-b883-a31d9070bf7e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:34.855598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c53d29-1d9a-4539-b883-a31d9070bf7e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:34.855884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c53d29-1d9a-4539-b883-a31d9070bf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:34.873976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-86c53d29-1d9a-4539-b883-a31d9070bf7e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:34.875161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-86c53d29-1d9a-4539-b883-a31d9070bf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:34.884623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 692/1384] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:34.926126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ae817-a501-446a-949a-45e16fdf1b39 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:34.941531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ae817-a501-446a-949a-45e16fdf1b39 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:34.947627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78456b93-54f4-46be-bb63-876df89cee18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:34.948510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78456b93-54f4-46be-bb63-876df89cee18 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:34.948988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78456b93-54f4-46be-bb63-876df89cee18 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:34.949461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78456b93-54f4-46be-bb63-876df89cee18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:34.958739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78456b93-54f4-46be-bb63-876df89cee18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:34.960581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78456b93-54f4-46be-bb63-876df89cee18 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:34.971304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 693/1385] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:37.123768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b6d1-2726-43af-9762-fb4816d3a46b None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:37.124252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b6d1-2726-43af-9762-fb4816d3a46b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:37.124358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b6d1-2726-43af-9762-fb4816d3a46b None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:37.125763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 693/1386] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:37.132093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eca0c0-210f-4682-87af-f74ccb5b91f9 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:37.132547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eca0c0-210f-4682-87af-f74ccb5b91f9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:37.133038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eca0c0-210f-4682-87af-f74ccb5b91f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:37.152814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-75eca0c0-210f-4682-87af-f74ccb5b91f9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:37.154059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-75eca0c0-210f-4682-87af-f74ccb5b91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:37.162253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 694/1387] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:37.211788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b6d1-2726-43af-9762-fb4816d3a46b None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:37.227568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b6d1-2726-43af-9762-fb4816d3a46b None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:37.233375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afcef5-8260-4be3-a396-03f6de751b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:37.234056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afcef5-8260-4be3-a396-03f6de751b13 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:37.234362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afcef5-8260-4be3-a396-03f6de751b13 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:37.234632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afcef5-8260-4be3-a396-03f6de751b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:37.243656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afcef5-8260-4be3-a396-03f6de751b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:37.245297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afcef5-8260-4be3-a396-03f6de751b13 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:37.255002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 694/1388] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:39.703938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b049ec-5c69-4eeb-8e9b-e16d6db60c77 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:39.704532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b049ec-5c69-4eeb-8e9b-e16d6db60c77 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:39.704532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b049ec-5c69-4eeb-8e9b-e16d6db60c77 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:39.705947 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 695/1389] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:39.712113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd95c-723c-4e09-a926-6c01e688ae6e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:39.712564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd95c-723c-4e09-a926-6c01e688ae6e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:39.712875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd95c-723c-4e09-a926-6c01e688ae6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:39.737262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ef1cd95c-723c-4e09-a926-6c01e688ae6e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:39.738782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ef1cd95c-723c-4e09-a926-6c01e688ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:39.750644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 695/1390] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:39.797282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b049ec-5c69-4eeb-8e9b-e16d6db60c77 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:39.815170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b049ec-5c69-4eeb-8e9b-e16d6db60c77 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:39.822136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be6e32-581e-4cc0-b440-9e668a42f044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:39.822852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be6e32-581e-4cc0-b440-9e668a42f044 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:39.823212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be6e32-581e-4cc0-b440-9e668a42f044 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:39.823533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be6e32-581e-4cc0-b440-9e668a42f044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:39.833746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be6e32-581e-4cc0-b440-9e668a42f044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:39.835572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be6e32-581e-4cc0-b440-9e668a42f044 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:39.846332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 696/1391] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:42.065541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f811-99e1-4a35-a4b2-7b0c56f4974b None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:42.066383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f811-99e1-4a35-a4b2-7b0c56f4974b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:42.066383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f811-99e1-4a35-a4b2-7b0c56f4974b None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:42.067639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 696/1392] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:42.073112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcceef-8f0f-4937-bfc1-b10253235ed7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:42.073588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcceef-8f0f-4937-bfc1-b10253235ed7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:42.073693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcceef-8f0f-4937-bfc1-b10253235ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:42.092714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1bbcceef-8f0f-4937-bfc1-b10253235ed7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:42.093967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1bbcceef-8f0f-4937-bfc1-b10253235ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:42.102993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 697/1393] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:42.150272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75f811-99e1-4a35-a4b2-7b0c56f4974b None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:42.165408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75f811-99e1-4a35-a4b2-7b0c56f4974b None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:42.171231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991d36b-5420-4bba-967c-23a49cf0447b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:42.171894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991d36b-5420-4bba-967c-23a49cf0447b service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:42.172180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991d36b-5420-4bba-967c-23a49cf0447b service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:42.172461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991d36b-5420-4bba-967c-23a49cf0447b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:42.181168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991d36b-5420-4bba-967c-23a49cf0447b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:42.182765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991d36b-5420-4bba-967c-23a49cf0447b service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:42.193370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 697/1394] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:44.246592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bc12-c0d6-49e3-9ef5-b21f2a02dba7 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.247145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bc12-c0d6-49e3-9ef5-b21f2a02dba7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.247145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bc12-c0d6-49e3-9ef5-b21f2a02dba7 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.248630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 698/1395] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:44.254020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76528365-290c-4f62-8347-bcd137eb40d3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.254539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76528365-290c-4f62-8347-bcd137eb40d3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.254539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76528365-290c-4f62-8347-bcd137eb40d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.274665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-76528365-290c-4f62-8347-bcd137eb40d3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:44.276037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-76528365-290c-4f62-8347-bcd137eb40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:44.284767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 698/1396] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:44:44.339870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afa65-c539-4feb-8d70-23f50c57f037 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.340135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afa65-c539-4feb-8d70-23f50c57f037 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.340483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afa65-c539-4feb-8d70-23f50c57f037 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.341862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 699/1397] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:44.342030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b50e-32b0-492a-83f7-a33c358eccdf None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.342324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b50e-32b0-492a-83f7-a33c358eccdf None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.342606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b50e-32b0-492a-83f7-a33c358eccdf None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.344028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 699/1398] 10.222.0.87 () {54 vars in 962 bytes} [Fri Jun 17 12:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:44:44.347344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317180-305e-46b3-bc5a-245bbd6316c2 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.347678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317180-305e-46b3-bc5a-245bbd6316c2 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.348020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317180-305e-46b3-bc5a-245bbd6316c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.349415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c917839-6e66-4cd7-b63e-0af5ce9480c7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.349696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c917839-6e66-4cd7-b63e-0af5ce9480c7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.349978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c917839-6e66-4cd7-b63e-0af5ce9480c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.367571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0e317180-305e-46b3-bc5a-245bbd6316c2 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:44.369640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0e317180-305e-46b3-bc5a-245bbd6316c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:44.378288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 700/1399] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:44.399669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0c917839-6e66-4cd7-b63e-0af5ce9480c7 None None] MFA Rules not processed for user `f733becb24b14cd5842de72b62d051b9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:44.400873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0c917839-6e66-4cd7-b63e-0af5ce9480c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:44.423549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e317180-305e-46b3-bc5a-245bbd6316c2 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:44.437473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e317180-305e-46b3-bc5a-245bbd6316c2 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:44.443300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cf183-29fc-497d-a1b4-81f440f84964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:44.444001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf183-29fc-497d-a1b4-81f440f84964 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.444290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf183-29fc-497d-a1b4-81f440f84964 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.444565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf183-29fc-497d-a1b4-81f440f84964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.455442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cf183-29fc-497d-a1b4-81f440f84964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:44.457043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cf183-29fc-497d-a1b4-81f440f84964 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:44.463234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 700/1400] 10.222.0.87 () {58 vars in 1059 bytes} [Fri Jun 17 12:44:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:44:44.467285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 701/1401] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:44.470997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c917839-6e66-4cd7-b63e-0af5ce9480c7 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:44.479236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c917839-6e66-4cd7-b63e-0af5ce9480c7 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:44.484138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5663131-5e25-4a0a-9dd0-4d07569718ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:44.484806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5663131-5e25-4a0a-9dd0-4d07569718ea service glance] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:44.485125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5663131-5e25-4a0a-9dd0-4d07569718ea service glance] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:44.485401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5663131-5e25-4a0a-9dd0-4d07569718ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:44.495870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5663131-5e25-4a0a-9dd0-4d07569718ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:44.497531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5663131-5e25-4a0a-9dd0-4d07569718ea service glance] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:44.508123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 701/1402] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:45.293028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af7d19-20ee-44dc-a1d5-9adcfc19bbe4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:45.293554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af7d19-20ee-44dc-a1d5-9adcfc19bbe4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:45.293714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af7d19-20ee-44dc-a1d5-9adcfc19bbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:45.348954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-32af7d19-20ee-44dc-a1d5-9adcfc19bbe4 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:45.350094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-32af7d19-20ee-44dc-a1d5-9adcfc19bbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:45.418819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 702/1403] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:44:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:45.459936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed71a-072d-4d97-87dd-c496730e6f2a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:45.460427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed71a-072d-4d97-87dd-c496730e6f2a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:45.460537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed71a-072d-4d97-87dd-c496730e6f2a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:45.462106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 702/1404] 10.222.0.87 () {54 vars in 942 bytes} [Fri Jun 17 12:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:45.468871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e348-ce98-43ad-a27d-da70111ca89d None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:45.469184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e348-ce98-43ad-a27d-da70111ca89d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:45.469467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e348-ce98-43ad-a27d-da70111ca89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:45.523496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a720e348-ce98-43ad-a27d-da70111ca89d None None] MFA Rules not processed for user `1a31e162778e4824b7bafd5dcd9b4b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:45.524836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a720e348-ce98-43ad-a27d-da70111ca89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:45.559110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:44:45.598351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 703/1405] 10.222.0.87 () {58 vars in 1039 bytes} [Fri Jun 17 12:44:45 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:45.607794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ed71a-072d-4d97-87dd-c496730e6f2a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:45.616203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:44:45.623489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ed71a-072d-4d97-87dd-c496730e6f2a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:45.628610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63876c8d-a8a7-4340-852e-dbb4e26fd42d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:45.629266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63876c8d-a8a7-4340-852e-dbb4e26fd42d service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:45.629549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63876c8d-a8a7-4340-852e-dbb4e26fd42d service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:45.629827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63876c8d-a8a7-4340-852e-dbb4e26fd42d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:45.638446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63876c8d-a8a7-4340-852e-dbb4e26fd42d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:45.640104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63876c8d-a8a7-4340-852e-dbb4e26fd42d service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:45.648222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 703/1406] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 12:44:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 12:44:46.771721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219bdc0-793e-4ba7-96f0-50bcd97c9001 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:46.772389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219bdc0-793e-4ba7-96f0-50bcd97c9001 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:46.772389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219bdc0-793e-4ba7-96f0-50bcd97c9001 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:46.773640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 704/1407] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:44:46.780738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca14c2-70c0-4cbc-8565-bb089325b93d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:46.781213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca14c2-70c0-4cbc-8565-bb089325b93d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:46.781332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca14c2-70c0-4cbc-8565-bb089325b93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:46.799787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-67ca14c2-70c0-4cbc-8565-bb089325b93d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:44:46.801012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-67ca14c2-70c0-4cbc-8565-bb089325b93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:44:46.809353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 704/1408] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:44:46.854553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219bdc0-793e-4ba7-96f0-50bcd97c9001 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:44:46.869954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219bdc0-793e-4ba7-96f0-50bcd97c9001 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:44:46.875434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271ff29-06ac-40f6-92d9-d253c113bc02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:44:46.876089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271ff29-06ac-40f6-92d9-d253c113bc02 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:44:46.876375 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271ff29-06ac-40f6-92d9-d253c113bc02 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:44:46.876656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271ff29-06ac-40f6-92d9-d253c113bc02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:44:46.885963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271ff29-06ac-40f6-92d9-d253c113bc02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:44:46.887580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271ff29-06ac-40f6-92d9-d253c113bc02 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:44:46.897237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 705/1409] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:44:56.794156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=8 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:44:56.893469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=21 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:45:08.424741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f293bb-649d-43b5-a802-9ac55fe3b527 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:08.425351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f293bb-649d-43b5-a802-9ac55fe3b527 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:08.425351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f293bb-649d-43b5-a802-9ac55fe3b527 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:08.426588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 705/1410] 10.222.0.87 () {54 vars in 918 bytes} [Fri Jun 17 12:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:08.432441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72333-3491-4f82-9387-f71c082af2af None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:08.432929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72333-3491-4f82-9387-f71c082af2af None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:08.433034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72333-3491-4f82-9387-f71c082af2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:08.457992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-90e72333-3491-4f82-9387-f71c082af2af None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:08.459233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-90e72333-3491-4f82-9387-f71c082af2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:08.468824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 706/1411] 10.222.0.87 () {58 vars in 1015 bytes} [Fri Jun 17 12:45:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:08.483676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfefb24-fa74-472a-8804-e0eeb9bf63b8 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:08.483970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfefb24-fa74-472a-8804-e0eeb9bf63b8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:08.484239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfefb24-fa74-472a-8804-e0eeb9bf63b8 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:08.485601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 706/1412] 10.222.0.87 () {54 vars in 918 bytes} [Fri Jun 17 12:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:08.490442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c6f2a-7df7-4a77-9a3e-e101271bcc2f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:08.490733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c6f2a-7df7-4a77-9a3e-e101271bcc2f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:08.491013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c6f2a-7df7-4a77-9a3e-e101271bcc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:08.510212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-e56c6f2a-7df7-4a77-9a3e-e101271bcc2f None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:08.511522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e56c6f2a-7df7-4a77-9a3e-e101271bcc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:08.521691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 707/1413] 10.222.0.87 () {58 vars in 1015 bytes} [Fri Jun 17 12:45:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:11.157311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529472a6-bb14-45c6-9937-fdb47cc7acb2 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:11.158167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529472a6-bb14-45c6-9937-fdb47cc7acb2 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:11.158167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529472a6-bb14-45c6-9937-fdb47cc7acb2 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:11.160597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:11.166845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1597e8-05cd-478f-950c-5e5a60e5ac67 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:11.167540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1597e8-05cd-478f-950c-5e5a60e5ac67 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:11.167745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1597e8-05cd-478f-950c-5e5a60e5ac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:11.191558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9b1597e8-05cd-478f-950c-5e5a60e5ac67 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:11.192771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9b1597e8-05cd-478f-950c-5e5a60e5ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:11.202108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:11.248531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529472a6-bb14-45c6-9937-fdb47cc7acb2 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:11.264645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529472a6-bb14-45c6-9937-fdb47cc7acb2 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:11.271241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f6fa0-de89-4b03-8198-55a720460c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:11.271926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f6fa0-de89-4b03-8198-55a720460c66 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:11.272221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f6fa0-de89-4b03-8198-55a720460c66 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:11.272501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f6fa0-de89-4b03-8198-55a720460c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:11.281478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f6fa0-de89-4b03-8198-55a720460c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:11.283087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f6fa0-de89-4b03-8198-55a720460c66 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:11.292634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 708/1416] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:12.256187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838100-fe42-4d35-9438-431602644700 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:12.258389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838100-fe42-4d35-9438-431602644700 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:12.258389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838100-fe42-4d35-9438-431602644700 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:12.258389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:12.264399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b44403-b5ee-4a94-bc82-d609a1268dbc None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:12.264843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b44403-b5ee-4a94-bc82-d609a1268dbc None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:12.264943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b44403-b5ee-4a94-bc82-d609a1268dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:12.286734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-96b44403-b5ee-4a94-bc82-d609a1268dbc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:12.288037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-96b44403-b5ee-4a94-bc82-d609a1268dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:12.296773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:12.461277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d838100-fe42-4d35-9438-431602644700 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:12.484455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d838100-fe42-4d35-9438-431602644700 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:12.490785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901df9d-de14-4f44-b165-0c1a0ba8b0f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:12.491495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901df9d-de14-4f44-b165-0c1a0ba8b0f6 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:12.491780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901df9d-de14-4f44-b165-0c1a0ba8b0f6 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:12.492054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901df9d-de14-4f44-b165-0c1a0ba8b0f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:12.500960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901df9d-de14-4f44-b165-0c1a0ba8b0f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:12.502535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901df9d-de14-4f44-b165-0c1a0ba8b0f6 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:12.512574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 710/1419] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:13.458392 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd26572-05d9-4593-8bfe-d3335dfbaeb0 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:13.459211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd26572-05d9-4593-8bfe-d3335dfbaeb0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:13.459211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd26572-05d9-4593-8bfe-d3335dfbaeb0 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:13.461395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:13.467067 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d12ba-7b8c-46a9-9b2e-47c1e6afcf8f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:13.467542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d12ba-7b8c-46a9-9b2e-47c1e6afcf8f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:13.467655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d12ba-7b8c-46a9-9b2e-47c1e6afcf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:13.487459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-146d12ba-7b8c-46a9-9b2e-47c1e6afcf8f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:13.488702 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-146d12ba-7b8c-46a9-9b2e-47c1e6afcf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:13.498860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:13.652520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd26572-05d9-4593-8bfe-d3335dfbaeb0 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:13.669030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd26572-05d9-4593-8bfe-d3335dfbaeb0 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:13.675016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c0c72-c43e-4a4b-b011-35758194323c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:13.675699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c0c72-c43e-4a4b-b011-35758194323c service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:13.675997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c0c72-c43e-4a4b-b011-35758194323c service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:13.676265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c0c72-c43e-4a4b-b011-35758194323c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:13.685547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c0c72-c43e-4a4b-b011-35758194323c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:13.687193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c0c72-c43e-4a4b-b011-35758194323c service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:13.697573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 711/1422] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:14.630121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d817eb-b308-4d6b-960a-f4282f507335 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:14.630691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d817eb-b308-4d6b-960a-f4282f507335 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:14.630817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d817eb-b308-4d6b-960a-f4282f507335 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:14.632909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:14.640351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2b717-da5d-480f-9077-bcd601b5aea9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:14.640743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2b717-da5d-480f-9077-bcd601b5aea9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:14.640976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2b717-da5d-480f-9077-bcd601b5aea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:14.670483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-08b2b717-da5d-480f-9077-bcd601b5aea9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:14.671717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-08b2b717-da5d-480f-9077-bcd601b5aea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:14.681385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:14.836975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d817eb-b308-4d6b-960a-f4282f507335 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:14.853542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d817eb-b308-4d6b-960a-f4282f507335 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:14.860242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83563d85-8c80-47a5-988c-b9a35cf263b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:14.860891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83563d85-8c80-47a5-988c-b9a35cf263b5 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:14.861169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83563d85-8c80-47a5-988c-b9a35cf263b5 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:14.861448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83563d85-8c80-47a5-988c-b9a35cf263b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:14.870555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83563d85-8c80-47a5-988c-b9a35cf263b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:14.872187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83563d85-8c80-47a5-988c-b9a35cf263b5 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:14.882508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 713/1425] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:15.786141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae6fb5-2e25-4ed3-862a-3258aa6d3913 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:15.787126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae6fb5-2e25-4ed3-862a-3258aa6d3913 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:15.787126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae6fb5-2e25-4ed3-862a-3258aa6d3913 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:15.789259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:15.794880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca838cf-1181-459d-8601-157f95bd2848 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:15.795334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca838cf-1181-459d-8601-157f95bd2848 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:15.795452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca838cf-1181-459d-8601-157f95bd2848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:15.816127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-bca838cf-1181-459d-8601-157f95bd2848 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:15.817347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-bca838cf-1181-459d-8601-157f95bd2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:15.828611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:15.982755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae6fb5-2e25-4ed3-862a-3258aa6d3913 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:16.008915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae6fb5-2e25-4ed3-862a-3258aa6d3913 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:16.016350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fd7c1-8bad-4107-a669-a5ab1ddb3826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:16.016998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fd7c1-8bad-4107-a669-a5ab1ddb3826 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:16.017283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fd7c1-8bad-4107-a669-a5ab1ddb3826 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:16.017563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fd7c1-8bad-4107-a669-a5ab1ddb3826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:16.026073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fd7c1-8bad-4107-a669-a5ab1ddb3826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:16.027714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fd7c1-8bad-4107-a669-a5ab1ddb3826 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:16.038473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 714/1428] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:16.908086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668c40-dc24-4035-8bea-312dbca6f832 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:16.909005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668c40-dc24-4035-8bea-312dbca6f832 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:16.909005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668c40-dc24-4035-8bea-312dbca6f832 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:16.911251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:16.916472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfc142-284d-4a53-a155-a3b86da603de None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:16.916930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfc142-284d-4a53-a155-a3b86da603de None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:16.917072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfc142-284d-4a53-a155-a3b86da603de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:16.944665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-fbbfc142-284d-4a53-a155-a3b86da603de None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:16.946397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fbbfc142-284d-4a53-a155-a3b86da603de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:16.956304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:17.111234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668c40-dc24-4035-8bea-312dbca6f832 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:17.129210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668c40-dc24-4035-8bea-312dbca6f832 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:17.135632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4d6bc-efe8-42b8-8f75-2f85f832abe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:17.136287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d6bc-efe8-42b8-8f75-2f85f832abe2 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:17.136572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d6bc-efe8-42b8-8f75-2f85f832abe2 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:17.136851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d6bc-efe8-42b8-8f75-2f85f832abe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:17.145292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4d6bc-efe8-42b8-8f75-2f85f832abe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:17.146885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4d6bc-efe8-42b8-8f75-2f85f832abe2 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:17.156594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 716/1431] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:18.009446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb88a-a340-4d32-8f59-21a85350ba97 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:18.010048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb88a-a340-4d32-8f59-21a85350ba97 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:18.010048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb88a-a340-4d32-8f59-21a85350ba97 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:18.011415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:18.019778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4308d8-ecc8-4741-93c8-8485d2b16c89 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:18.020322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4308d8-ecc8-4741-93c8-8485d2b16c89 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:18.020421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4308d8-ecc8-4741-93c8-8485d2b16c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:18.042126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7b4308d8-ecc8-4741-93c8-8485d2b16c89 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:18.043342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7b4308d8-ecc8-4741-93c8-8485d2b16c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:18.052945 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:18.208260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabb88a-a340-4d32-8f59-21a85350ba97 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:18.232906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabb88a-a340-4d32-8f59-21a85350ba97 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:18.239924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d16fe-54b8-4181-ae4d-fd967234d3ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:18.240566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d16fe-54b8-4181-ae4d-fd967234d3ac service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:18.240851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d16fe-54b8-4181-ae4d-fd967234d3ac service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:18.241134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d16fe-54b8-4181-ae4d-fd967234d3ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:18.249644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d16fe-54b8-4181-ae4d-fd967234d3ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:18.251246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d16fe-54b8-4181-ae4d-fd967234d3ac service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:18.260957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 717/1434] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:19.112174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca39ac-711f-4717-b3a5-9996ba288501 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:19.112646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca39ac-711f-4717-b3a5-9996ba288501 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:19.112761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca39ac-711f-4717-b3a5-9996ba288501 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:19.114149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:19.119637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aaa2a-ed78-4e37-8392-1f1095b17a2a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:19.120086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aaa2a-ed78-4e37-8392-1f1095b17a2a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:19.120197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aaa2a-ed78-4e37-8392-1f1095b17a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:19.144266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-673aaa2a-ed78-4e37-8392-1f1095b17a2a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:19.145475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-673aaa2a-ed78-4e37-8392-1f1095b17a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:19.153869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:19.305720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca39ac-711f-4717-b3a5-9996ba288501 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:19.324509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca39ac-711f-4717-b3a5-9996ba288501 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:19.331987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f640d2-4f6f-4283-968b-846e7b341415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:19.332634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f640d2-4f6f-4283-968b-846e7b341415 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:19.332919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f640d2-4f6f-4283-968b-846e7b341415 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:19.333197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f640d2-4f6f-4283-968b-846e7b341415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:19.342884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f640d2-4f6f-4283-968b-846e7b341415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:19.345255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f640d2-4f6f-4283-968b-846e7b341415 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:19.360770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 719/1437] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:20.214640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977846c-159a-4dd5-bc66-a1ec43582ac9 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:20.215246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977846c-159a-4dd5-bc66-a1ec43582ac9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:20.215246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977846c-159a-4dd5-bc66-a1ec43582ac9 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:20.217429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:20.225578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f034162-3be8-4e46-8f4c-c6aa591edf59 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:20.226314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f034162-3be8-4e46-8f4c-c6aa591edf59 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:20.226470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f034162-3be8-4e46-8f4c-c6aa591edf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:20.251674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8f034162-3be8-4e46-8f4c-c6aa591edf59 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:20.252876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8f034162-3be8-4e46-8f4c-c6aa591edf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:20.261312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:45:20.419415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977846c-159a-4dd5-bc66-a1ec43582ac9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:20.434731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977846c-159a-4dd5-bc66-a1ec43582ac9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:20.440792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df723656-8031-4ede-9f52-a0354310757c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:20.441444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df723656-8031-4ede-9f52-a0354310757c service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:20.441730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df723656-8031-4ede-9f52-a0354310757c service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:20.442007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df723656-8031-4ede-9f52-a0354310757c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:20.451094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df723656-8031-4ede-9f52-a0354310757c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:20.452741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df723656-8031-4ede-9f52-a0354310757c service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:20.462806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 720/1440] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:22.476113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc71f9a-4850-46b8-8a47-aa956a1f4da1 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:22.476673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc71f9a-4850-46b8-8a47-aa956a1f4da1 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:22.476673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc71f9a-4850-46b8-8a47-aa956a1f4da1 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:22.477978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:22.484595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ca45-ad1a-463a-bc2f-0323df450177 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:22.485333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ca45-ad1a-463a-bc2f-0323df450177 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:22.485482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ca45-ad1a-463a-bc2f-0323df450177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:22.508305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-00f4ca45-ad1a-463a-bc2f-0323df450177 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:22.509530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-00f4ca45-ad1a-463a-bc2f-0323df450177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:22.519175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:22.558773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc71f9a-4850-46b8-8a47-aa956a1f4da1 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:22.577495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc71f9a-4850-46b8-8a47-aa956a1f4da1 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:22.584630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fb7ca-1779-4a6c-84b2-afdd95db4f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:22.585545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb7ca-1779-4a6c-84b2-afdd95db4f61 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:22.585901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb7ca-1779-4a6c-84b2-afdd95db4f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:22.586261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb7ca-1779-4a6c-84b2-afdd95db4f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:22.597431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fb7ca-1779-4a6c-84b2-afdd95db4f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:22.598881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fb7ca-1779-4a6c-84b2-afdd95db4f61 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:22.608648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 722/1443] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:24.935212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b344e9-6184-4eb7-a930-5ea071dfd9e9 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:24.936160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b344e9-6184-4eb7-a930-5ea071dfd9e9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:24.936160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b344e9-6184-4eb7-a930-5ea071dfd9e9 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:24.938289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:24.947237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e39b02-1e33-4b01-befa-36e821947df7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:24.947729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e39b02-1e33-4b01-befa-36e821947df7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:24.947830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e39b02-1e33-4b01-befa-36e821947df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:24.971139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-40e39b02-1e33-4b01-befa-36e821947df7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:24.972373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-40e39b02-1e33-4b01-befa-36e821947df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:24.981007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:45:25.032239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b344e9-6184-4eb7-a930-5ea071dfd9e9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:25.034347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-37b344e9-6184-4eb7-a930-5ea071dfd9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:25.049279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b344e9-6184-4eb7-a930-5ea071dfd9e9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:25.053282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12da3cf-7648-4b77-965d-139f76c030ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:25.053929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12da3cf-7648-4b77-965d-139f76c030ab service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:25.054213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12da3cf-7648-4b77-965d-139f76c030ab service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:25.054489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12da3cf-7648-4b77-965d-139f76c030ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:25.063917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12da3cf-7648-4b77-965d-139f76c030ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:25.065720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12da3cf-7648-4b77-965d-139f76c030ab service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:25.075131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 723/1446] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:27.591105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdddd6d-d04f-4a23-beeb-2687324e5b0d None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:27.591676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdddd6d-d04f-4a23-beeb-2687324e5b0d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:27.591676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdddd6d-d04f-4a23-beeb-2687324e5b0d None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:27.593897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:27.602009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1383-aeab-4974-8e72-2b0de38a8f26 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:27.602449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1383-aeab-4974-8e72-2b0de38a8f26 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:27.602584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1383-aeab-4974-8e72-2b0de38a8f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:27.622924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-db3d1383-aeab-4974-8e72-2b0de38a8f26 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:27.624050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-db3d1383-aeab-4974-8e72-2b0de38a8f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:27.632939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:45:27.675046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdddd6d-d04f-4a23-beeb-2687324e5b0d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:27.690720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdddd6d-d04f-4a23-beeb-2687324e5b0d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:27.694921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144fcc8-bfda-45ff-8f52-c70813a6345e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:27.695619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144fcc8-bfda-45ff-8f52-c70813a6345e service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:27.696036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144fcc8-bfda-45ff-8f52-c70813a6345e service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:27.696320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144fcc8-bfda-45ff-8f52-c70813a6345e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:27.706207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144fcc8-bfda-45ff-8f52-c70813a6345e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:27.707674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144fcc8-bfda-45ff-8f52-c70813a6345e service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:27.717693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 725/1449] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:30.011341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7789d1-1bec-40f8-b293-a74c95df447e None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:30.012021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7789d1-1bec-40f8-b293-a74c95df447e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:30.012393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7789d1-1bec-40f8-b293-a74c95df447e None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:30.016196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:30.025449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa4a9-40c5-4025-a4ef-c188f094149c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:30.025921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa4a9-40c5-4025-a4ef-c188f094149c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:30.026021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa4a9-40c5-4025-a4ef-c188f094149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:30.055143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-76eaa4a9-40c5-4025-a4ef-c188f094149c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:30.056982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-76eaa4a9-40c5-4025-a4ef-c188f094149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:30.067895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:30.114496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7789d1-1bec-40f8-b293-a74c95df447e None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:30.127160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7789d1-1bec-40f8-b293-a74c95df447e None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:30.131541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841a2a6-a0dd-47a5-8657-6687ee27c729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:30.132204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841a2a6-a0dd-47a5-8657-6687ee27c729 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:30.132526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841a2a6-a0dd-47a5-8657-6687ee27c729 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:30.132804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841a2a6-a0dd-47a5-8657-6687ee27c729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:30.142612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841a2a6-a0dd-47a5-8657-6687ee27c729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:30.144102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841a2a6-a0dd-47a5-8657-6687ee27c729 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:30.153958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 726/1452] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:32.315872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53c7ff-f9d8-4866-8a2d-d1f6cac13825 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:32.316788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53c7ff-f9d8-4866-8a2d-d1f6cac13825 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:32.316788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53c7ff-f9d8-4866-8a2d-d1f6cac13825 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:32.319018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:32.328338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce114a-a28c-49d7-b561-91e73b24bfb4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:32.329131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce114a-a28c-49d7-b561-91e73b24bfb4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:32.329270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce114a-a28c-49d7-b561-91e73b24bfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:32.351876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d8ce114a-a28c-49d7-b561-91e73b24bfb4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:32.353089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d8ce114a-a28c-49d7-b561-91e73b24bfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:32.362518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:32.402297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53c7ff-f9d8-4866-8a2d-d1f6cac13825 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:32.414935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53c7ff-f9d8-4866-8a2d-d1f6cac13825 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:32.419135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d58ef-00b6-42c6-addd-7daa938ed5cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:32.419815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d58ef-00b6-42c6-addd-7daa938ed5cc service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:32.420097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d58ef-00b6-42c6-addd-7daa938ed5cc service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:32.420363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d58ef-00b6-42c6-addd-7daa938ed5cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:32.429632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d58ef-00b6-42c6-addd-7daa938ed5cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:32.431092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d58ef-00b6-42c6-addd-7daa938ed5cc service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:32.441257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 728/1455] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:34.666781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d656-12e8-4222-8ca2-6c78820b7a4f None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:34.667414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d656-12e8-4222-8ca2-6c78820b7a4f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:34.667414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d656-12e8-4222-8ca2-6c78820b7a4f None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:34.669330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:34.675080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e082ea-2f38-406a-b3fa-02c29d565d74 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:34.675546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e082ea-2f38-406a-b3fa-02c29d565d74 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:34.675666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e082ea-2f38-406a-b3fa-02c29d565d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:34.695241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f7e082ea-2f38-406a-b3fa-02c29d565d74 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:34.696469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f7e082ea-2f38-406a-b3fa-02c29d565d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:34.705340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:34.744019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554d656-12e8-4222-8ca2-6c78820b7a4f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:34.756688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554d656-12e8-4222-8ca2-6c78820b7a4f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:34.760486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c365ea-69f3-46b6-adad-9b80ee6e798c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:34.761116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c365ea-69f3-46b6-adad-9b80ee6e798c service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:34.761658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c365ea-69f3-46b6-adad-9b80ee6e798c service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:34.761931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c365ea-69f3-46b6-adad-9b80ee6e798c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:34.771560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c365ea-69f3-46b6-adad-9b80ee6e798c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:34.773052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c365ea-69f3-46b6-adad-9b80ee6e798c service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:34.783142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 729/1458] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:36.967850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e7817-5980-4368-a519-2ad1c8f9a450 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:36.968439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e7817-5980-4368-a519-2ad1c8f9a450 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:36.968439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e7817-5980-4368-a519-2ad1c8f9a450 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:36.969781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:36.976112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226133a-e5a0-4efc-823c-ff91bde92a52 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:36.976568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226133a-e5a0-4efc-823c-ff91bde92a52 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:36.976718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226133a-e5a0-4efc-823c-ff91bde92a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:36.985097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=70 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:45:36.998827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c226133a-e5a0-4efc-823c-ff91bde92a52 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:37.000035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c226133a-e5a0-4efc-823c-ff91bde92a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:37.008487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:37.046192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e7817-5980-4368-a519-2ad1c8f9a450 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:37.055533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=73 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:45:37.064589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e7817-5980-4368-a519-2ad1c8f9a450 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:37.071173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd520801-1116-4fb5-b805-4aef713f79d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:37.072195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd520801-1116-4fb5-b805-4aef713f79d6 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:37.072616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd520801-1116-4fb5-b805-4aef713f79d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:37.073052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd520801-1116-4fb5-b805-4aef713f79d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:37.083607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd520801-1116-4fb5-b805-4aef713f79d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:37.085063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd520801-1116-4fb5-b805-4aef713f79d6 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:37.095118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 731/1461] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:39.275636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cd3d-5db6-46d5-bcc2-ed113488ea19 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:39.276503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cd3d-5db6-46d5-bcc2-ed113488ea19 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:39.277065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cd3d-5db6-46d5-bcc2-ed113488ea19 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:39.278624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:39.284175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac930f-39ae-4c47-8df1-da19ea8aa70e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:39.284638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac930f-39ae-4c47-8df1-da19ea8aa70e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:39.284743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac930f-39ae-4c47-8df1-da19ea8aa70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:39.303355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a9ac930f-39ae-4c47-8df1-da19ea8aa70e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:39.304642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a9ac930f-39ae-4c47-8df1-da19ea8aa70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:39.313855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:39.351171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cd3d-5db6-46d5-bcc2-ed113488ea19 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:39.362380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cd3d-5db6-46d5-bcc2-ed113488ea19 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:39.365993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10256f7-c694-4c59-b085-b305a9a89db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:39.366850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10256f7-c694-4c59-b085-b305a9a89db0 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:39.367198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10256f7-c694-4c59-b085-b305a9a89db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:39.367568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10256f7-c694-4c59-b085-b305a9a89db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:39.377521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10256f7-c694-4c59-b085-b305a9a89db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:39.378972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10256f7-c694-4c59-b085-b305a9a89db0 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:39.388913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 732/1464] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:41.511675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc57b6-7f64-4bc5-82f9-b76ba6175c3a None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:41.512640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc57b6-7f64-4bc5-82f9-b76ba6175c3a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:41.512640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc57b6-7f64-4bc5-82f9-b76ba6175c3a None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:41.514940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:41.521346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af705971-f5df-4be6-aeaf-77df9ee402ed None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:41.521811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af705971-f5df-4be6-aeaf-77df9ee402ed None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:41.521911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af705971-f5df-4be6-aeaf-77df9ee402ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:41.549504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-af705971-f5df-4be6-aeaf-77df9ee402ed None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:41.550715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-af705971-f5df-4be6-aeaf-77df9ee402ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:41.560414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:41.597247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc57b6-7f64-4bc5-82f9-b76ba6175c3a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:41.612951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc57b6-7f64-4bc5-82f9-b76ba6175c3a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:41.617417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779d5bc-0cda-4eca-b4e5-ae7ef79ac63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:41.618054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d5bc-0cda-4eca-b4e5-ae7ef79ac63c service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:41.618337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d5bc-0cda-4eca-b4e5-ae7ef79ac63c service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:41.618606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d5bc-0cda-4eca-b4e5-ae7ef79ac63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:41.628454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779d5bc-0cda-4eca-b4e5-ae7ef79ac63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:41.629881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779d5bc-0cda-4eca-b4e5-ae7ef79ac63c service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:41.639659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 734/1467] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:43.818592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9197592-66f5-404f-87cf-58ff29a04944 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:43.819199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9197592-66f5-404f-87cf-58ff29a04944 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:43.819199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9197592-66f5-404f-87cf-58ff29a04944 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:43.820605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:45:43.827251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed5e2-ae6c-4ba7-8586-d3b0301c5f3b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:43.828030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed5e2-ae6c-4ba7-8586-d3b0301c5f3b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:43.828129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed5e2-ae6c-4ba7-8586-d3b0301c5f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:43.857218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-79bed5e2-ae6c-4ba7-8586-d3b0301c5f3b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:45:43.858453 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-79bed5e2-ae6c-4ba7-8586-d3b0301c5f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:45:43.867468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:45:43.907981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9197592-66f5-404f-87cf-58ff29a04944 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:45:43.921829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9197592-66f5-404f-87cf-58ff29a04944 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:45:43.925785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14d119-ec70-4dd4-b76e-b8173bcc4403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:45:43.926422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14d119-ec70-4dd4-b76e-b8173bcc4403 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:45:43.926701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14d119-ec70-4dd4-b76e-b8173bcc4403 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:45:43.926979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14d119-ec70-4dd4-b76e-b8173bcc4403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:45:43.936571 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14d119-ec70-4dd4-b76e-b8173bcc4403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:45:43.938001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14d119-ec70-4dd4-b76e-b8173bcc4403 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:45:43.947737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 735/1470] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:45:53.848240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=17 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:45:53.943595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=17 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:46:04.012155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f114b-c164-43c4-bf06-8f7819ece7c3 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:04.012714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f114b-c164-43c4-bf06-8f7819ece7c3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:04.012714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f114b-c164-43c4-bf06-8f7819ece7c3 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:04.014100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 736/1471] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:04.019524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23d2d-0eab-4e4a-b441-bad5684612f4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:04.019983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23d2d-0eab-4e4a-b441-bad5684612f4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:04.020086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23d2d-0eab-4e4a-b441-bad5684612f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:04.039637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0de23d2d-0eab-4e4a-b441-bad5684612f4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:04.040867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0de23d2d-0eab-4e4a-b441-bad5684612f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:04.050516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 736/1472] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:46:04.098884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e33b2-a771-46e6-8b2c-c87cd53aa1bd None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:04.099177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e33b2-a771-46e6-8b2c-c87cd53aa1bd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:04.099459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e33b2-a771-46e6-8b2c-c87cd53aa1bd None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:04.100837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 737/1473] 10.222.0.87 () {54 vars in 962 bytes} [Fri Jun 17 12:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:04.105844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1329f5-81b7-4745-bbd4-61a3e406b364 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:04.106127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1329f5-81b7-4745-bbd4-61a3e406b364 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:04.106406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1329f5-81b7-4745-bbd4-61a3e406b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:04.126269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8c1329f5-81b7-4745-bbd4-61a3e406b364 None None] MFA Rules not processed for user `f733becb24b14cd5842de72b62d051b9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:04.127507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8c1329f5-81b7-4745-bbd4-61a3e406b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:04.134421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 737/1474] 10.222.0.87 () {58 vars in 1059 bytes} [Fri Jun 17 12:46:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:46:04.142195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e33b2-a771-46e6-8b2c-c87cd53aa1bd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:46:04.155940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e33b2-a771-46e6-8b2c-c87cd53aa1bd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:46:04.160674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545b9c3-f5f2-4844-b343-6d1b042ba360 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:46:04.161301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b9c3-f5f2-4844-b343-6d1b042ba360 service glance] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:04.161587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b9c3-f5f2-4844-b343-6d1b042ba360 service glance] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:04.161864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b9c3-f5f2-4844-b343-6d1b042ba360 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:04.171884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545b9c3-f5f2-4844-b343-6d1b042ba360 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:46:04.173463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545b9c3-f5f2-4844-b343-6d1b042ba360 service glance] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:46:04.183509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 738/1475] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:46:06.392721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467609c1-5173-487b-843b-6ff1f6239cb4 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:06.393277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467609c1-5173-487b-843b-6ff1f6239cb4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:06.393277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467609c1-5173-487b-843b-6ff1f6239cb4 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:06.394611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 738/1476] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:06.399960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823912c7-7d07-4d3e-86ac-eaf93158a498 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:06.400443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823912c7-7d07-4d3e-86ac-eaf93158a498 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:06.400550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823912c7-7d07-4d3e-86ac-eaf93158a498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:06.419969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-823912c7-7d07-4d3e-86ac-eaf93158a498 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:06.421172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-823912c7-7d07-4d3e-86ac-eaf93158a498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:06.429772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 739/1477] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:46:06.453876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467609c1-5173-487b-843b-6ff1f6239cb4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:46:06.469251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467609c1-5173-487b-843b-6ff1f6239cb4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:46:06.474539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b90735-e92a-4b2e-b9db-ee1be0de4c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:46:06.475202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b90735-e92a-4b2e-b9db-ee1be0de4c98 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:06.475547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b90735-e92a-4b2e-b9db-ee1be0de4c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:06.475825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b90735-e92a-4b2e-b9db-ee1be0de4c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:06.476576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b90735-e92a-4b2e-b9db-ee1be0de4c98 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:46:06.478390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-47b90735-e92a-4b2e-b9db-ee1be0de4c98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:46:06.478390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:46:06.478390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:46:06.479545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b90735-e92a-4b2e-b9db-ee1be0de4c98 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:46:06.498976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 739/1478] 10.222.0.87 () {56 vars in 1108 bytes} [Fri Jun 17 12:46:06 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:46:08.692984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a9395-86ff-456a-8141-de33309a0f30 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:08.693639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a9395-86ff-456a-8141-de33309a0f30 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:08.693639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a9395-86ff-456a-8141-de33309a0f30 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:08.694961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 740/1479] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:08.700717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b3d08-2066-44e8-a6d9-32de5c4f214f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:08.701219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b3d08-2066-44e8-a6d9-32de5c4f214f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:08.701219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b3d08-2066-44e8-a6d9-32de5c4f214f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:08.722620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-332b3d08-2066-44e8-a6d9-32de5c4f214f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:08.723876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-332b3d08-2066-44e8-a6d9-32de5c4f214f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:08.733742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 740/1480] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:46:08.785895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a9395-86ff-456a-8141-de33309a0f30 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:46:08.802767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a9395-86ff-456a-8141-de33309a0f30 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:46:08.809273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1033b3b-860b-4e5c-a886-702275e23555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:46:08.809896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033b3b-860b-4e5c-a886-702275e23555 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:08.810210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033b3b-860b-4e5c-a886-702275e23555 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:08.810457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033b3b-860b-4e5c-a886-702275e23555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:08.819278 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1033b3b-860b-4e5c-a886-702275e23555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:46:08.820896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1033b3b-860b-4e5c-a886-702275e23555 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:46:08.831096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 741/1481] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:46:11.153127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b991d0-bb6e-46ee-bafc-03a0e2f8a9b8 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:11.153662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b991d0-bb6e-46ee-bafc-03a0e2f8a9b8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:11.153662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b991d0-bb6e-46ee-bafc-03a0e2f8a9b8 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:11.155058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 741/1482] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:11.160482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc0fdc-9a80-4738-9d6f-5e5759c878bb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:11.160934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc0fdc-9a80-4738-9d6f-5e5759c878bb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:11.161039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc0fdc-9a80-4738-9d6f-5e5759c878bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:11.179091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7acc0fdc-9a80-4738-9d6f-5e5759c878bb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:11.180319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7acc0fdc-9a80-4738-9d6f-5e5759c878bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:11.188763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 742/1483] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:46:11.229558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b991d0-bb6e-46ee-bafc-03a0e2f8a9b8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:46:11.244279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b991d0-bb6e-46ee-bafc-03a0e2f8a9b8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:46:11.248905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf6ec-d989-4c3d-9348-47b1ed2100ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:46:11.249551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf6ec-d989-4c3d-9348-47b1ed2100ca service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:11.249867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf6ec-d989-4c3d-9348-47b1ed2100ca service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:11.250145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf6ec-d989-4c3d-9348-47b1ed2100ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:11.261357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf6ec-d989-4c3d-9348-47b1ed2100ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:46:11.262818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf6ec-d989-4c3d-9348-47b1ed2100ca service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:46:11.273120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 742/1484] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:46:13.479187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c52f5-d69f-4850-986d-d8d4b41c0d03 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:13.479777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c52f5-d69f-4850-986d-d8d4b41c0d03 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:13.479777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c52f5-d69f-4850-986d-d8d4b41c0d03 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:13.481666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 743/1485] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:13.487350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e7da-0774-4d15-bcc9-37f4d240f65a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:13.487819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e7da-0774-4d15-bcc9-37f4d240f65a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:13.487942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e7da-0774-4d15-bcc9-37f4d240f65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:13.506157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4d34e7da-0774-4d15-bcc9-37f4d240f65a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:13.507346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4d34e7da-0774-4d15-bcc9-37f4d240f65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:13.515673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 743/1486] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:46:13.553633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c52f5-d69f-4850-986d-d8d4b41c0d03 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:46:13.566636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c52f5-d69f-4850-986d-d8d4b41c0d03 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:46:13.571712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62aea83-baab-4bb7-b5c8-aabaec180c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:46:13.572358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aea83-baab-4bb7-b5c8-aabaec180c4e service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:13.572644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aea83-baab-4bb7-b5c8-aabaec180c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:13.572917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aea83-baab-4bb7-b5c8-aabaec180c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:13.582799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62aea83-baab-4bb7-b5c8-aabaec180c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:46:13.584284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62aea83-baab-4bb7-b5c8-aabaec180c4e service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:46:13.593844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 744/1487] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:46:15.894414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b923f-b0ae-40fc-aa27-52209633abc6 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:15.895020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b923f-b0ae-40fc-aa27-52209633abc6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:15.895020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b923f-b0ae-40fc-aa27-52209633abc6 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:15.896380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 744/1488] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:15.902889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf181-20be-4cc6-96fc-e4db72364427 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:15.903688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf181-20be-4cc6-96fc-e4db72364427 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:15.903875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf181-20be-4cc6-96fc-e4db72364427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:15.929600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ccebf181-20be-4cc6-96fc-e4db72364427 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:15.930809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ccebf181-20be-4cc6-96fc-e4db72364427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:15.940949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 745/1489] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:46:15.982120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b923f-b0ae-40fc-aa27-52209633abc6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:46:15.999026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b923f-b0ae-40fc-aa27-52209633abc6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:46:16.003908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fd5a1-38ef-4e45-8640-bdb5fa97617c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:46:16.004555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fd5a1-38ef-4e45-8640-bdb5fa97617c service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:16.004837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fd5a1-38ef-4e45-8640-bdb5fa97617c service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:16.005111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fd5a1-38ef-4e45-8640-bdb5fa97617c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:16.014633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fd5a1-38ef-4e45-8640-bdb5fa97617c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:46:16.016120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fd5a1-38ef-4e45-8640-bdb5fa97617c service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:46:16.025711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 745/1490] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:46:18.373628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844d20-8ed3-4cde-8239-8121cb51599f None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:18.374510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844d20-8ed3-4cde-8239-8121cb51599f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:18.374510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844d20-8ed3-4cde-8239-8121cb51599f None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:18.376939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 746/1491] 10.222.0.87 () {54 vars in 926 bytes} [Fri Jun 17 12:46:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:46:18.383012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffbf98-0e50-416e-8b5d-8309cb06b458 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:18.383475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffbf98-0e50-416e-8b5d-8309cb06b458 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:18.383640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffbf98-0e50-416e-8b5d-8309cb06b458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:18.404574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-edffbf98-0e50-416e-8b5d-8309cb06b458 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:46:18.405786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-edffbf98-0e50-416e-8b5d-8309cb06b458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:46:18.415187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 746/1492] 10.222.0.87 () {58 vars in 1023 bytes} [Fri Jun 17 12:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:46:18.457785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d844d20-8ed3-4cde-8239-8121cb51599f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:46:18.470981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d844d20-8ed3-4cde-8239-8121cb51599f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:46:18.475166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5395102-ea6e-43fe-9f08-baded8edcb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:46:18.475853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395102-ea6e-43fe-9f08-baded8edcb95 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:46:18.476141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395102-ea6e-43fe-9f08-baded8edcb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:46:18.476422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395102-ea6e-43fe-9f08-baded8edcb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:46:18.485846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5395102-ea6e-43fe-9f08-baded8edcb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:46:18.487484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5395102-ea6e-43fe-9f08-baded8edcb95 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:46:18.497093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 747/1493] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:46:28.399136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=33 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:46:28.492895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=33 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:47:43.187198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750198e-66ad-479b-be97-266a187e2a7d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.188165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750198e-66ad-479b-be97-266a187e2a7d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.188165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750198e-66ad-479b-be97-266a187e2a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.210234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-6750198e-66ad-479b-be97-266a187e2a7d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:47:43.211475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6750198e-66ad-479b-be97-266a187e2a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:47:43.221176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 747/1494] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:47:43.231539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27d20e-1738-4ccb-8ea9-8a61ca6197f4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.232073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27d20e-1738-4ccb-8ea9-8a61ca6197f4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.232073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27d20e-1738-4ccb-8ea9-8a61ca6197f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.255222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9d27d20e-1738-4ccb-8ea9-8a61ca6197f4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:47:43.256452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9d27d20e-1738-4ccb-8ea9-8a61ca6197f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:47:43.266133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 748/1495] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:47:43.270904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750198e-66ad-479b-be97-266a187e2a7d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.281786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750198e-66ad-479b-be97-266a187e2a7d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:43.287323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:43.288014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.288306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.288582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.289316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:47:43.289625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:43.290710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:43.290710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:47:43.290710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:47:43.291584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:43.303316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 748/1496] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:47:43.311859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27d20e-1738-4ccb-8ea9-8a61ca6197f4 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.321144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27d20e-1738-4ccb-8ea9-8a61ca6197f4 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:43.326630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:43.327282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.327609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.327880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.328576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:43.330342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING py.warnings [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:43.330342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: warnings.warn(msg) Jun 17 12:47:43.330342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: Jun 17 12:47:43.331586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:43.370723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 749/1497] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:47:43 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:47:43.375006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.385033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eeb834-3988-4273-8a06-c9edb10ab4b1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:43.451119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:43.451836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.452138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.452414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.453122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:43.454162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:43.454162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:47:43.454162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:47:43.455409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:43.477846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 749/1498] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:47:43 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:47:43.482181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.492608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d644338-a1c2-4261-9a75-70734e30fa59 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:43.497864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:43.498534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.498821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.499099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.499859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:43.500887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING py.warnings [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:43.500887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: warnings.warn(msg) Jun 17 12:47:43.500887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: Jun 17 12:47:43.501777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:43.515505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 750/1499] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:47:43.519769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.530447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cd1c0-9804-4d48-8c1f-f65b1b78afbd admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:43.536529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731083b-4fe3-46ec-ac61-b080ec44acee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:43.537199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731083b-4fe3-46ec-ac61-b080ec44acee admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.537501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731083b-4fe3-46ec-ac61-b080ec44acee admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.537766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731083b-4fe3-46ec-ac61-b080ec44acee admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.538491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731083b-4fe3-46ec-ac61-b080ec44acee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:43.539557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-e731083b-4fe3-46ec-ac61-b080ec44acee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:43.539557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:47:43.539557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:47:43.540467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731083b-4fe3-46ec-ac61-b080ec44acee admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:43.554222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 750/1500] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:47:43.559173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.569846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c57d75-e3c4-4788-943e-3b2b85836ad3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:43.575955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80bc54-dcf9-4dc8-b92e-91df4bdd6672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:43.576856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80bc54-dcf9-4dc8-b92e-91df4bdd6672 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.577138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80bc54-dcf9-4dc8-b92e-91df4bdd6672 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.577411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80bc54-dcf9-4dc8-b92e-91df4bdd6672 admin admin] PATH_INFO: `/v3/projects/79bc762c6681485f9c84513470091d87/users/18ea3d7882ee4c3dbf3f58082584e012/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.593238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80bc54-dcf9-4dc8-b92e-91df4bdd6672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bc762c6681485f9c84513470091d87, user_id=18ea3d7882ee4c3dbf3f58082584e012, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:43.594277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING py.warnings [None req-7d80bc54-dcf9-4dc8-b92e-91df4bdd6672 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:43.594277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: warnings.warn(msg) Jun 17 12:47:43.594277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: Jun 17 12:47:43.596251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80bc54-dcf9-4dc8-b92e-91df4bdd6672 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:43.610683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 751/1501] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:47:43 2022] PUT /identity/v3/projects/79bc762c6681485f9c84513470091d87/users/18ea3d7882ee4c3dbf3f58082584e012/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:47:43.622309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e079884-7bb1-4450-b7d5-593554608574 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.622539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e079884-7bb1-4450-b7d5-593554608574 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.622812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e079884-7bb1-4450-b7d5-593554608574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.656960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-7e079884-7bb1-4450-b7d5-593554608574 None None] MFA Rules not processed for user `18ea3d7882ee4c3dbf3f58082584e012`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:47:43.658210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7e079884-7bb1-4450-b7d5-593554608574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:47:43.734718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 751/1502] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:47:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:47:43.748554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80619d-fe33-48ab-aa8b-1d520d665623 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.748850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80619d-fe33-48ab-aa8b-1d520d665623 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.749157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80619d-fe33-48ab-aa8b-1d520d665623 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.750559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 752/1503] 10.222.0.87 () {54 vars in 963 bytes} [Fri Jun 17 12:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:47:43.755601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e67c7-8dfb-4841-8202-7355160886b4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.755899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e67c7-8dfb-4841-8202-7355160886b4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.756204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e67c7-8dfb-4841-8202-7355160886b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.775156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-293e67c7-8dfb-4841-8202-7355160886b4 None None] MFA Rules not processed for user `1315c22b7db84afd9ef41bf853b9819b`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:47:43.776405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-293e67c7-8dfb-4841-8202-7355160886b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:47:43.822088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:47:43.831784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 752/1504] 10.222.0.87 () {58 vars in 1060 bytes} [Fri Jun 17 12:47:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:47:43.840020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80619d-fe33-48ab-aa8b-1d520d665623 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.851941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80619d-fe33-48ab-aa8b-1d520d665623 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:43.856445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:43.857091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.857372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.857648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.866200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:43.867655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:43.876938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 753/1505] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:47:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:47:43.922288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3426d-e59e-4742-9bc0-6160cc8835c7 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:43.922579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3426d-e59e-4742-9bc0-6160cc8835c7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:43.922856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3426d-e59e-4742-9bc0-6160cc8835c7 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:43.924186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 753/1506] 10.222.0.87 () {52 vars in 829 bytes} [Fri Jun 17 12:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:47:43.928115 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:43.995651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0194b5-359d-4113-832b-da365cd460ff service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:44.000620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:44.001495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:44.001781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:44.002048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] PATH_INFO: `/v3/users/18ea3d7882ee4c3dbf3f58082584e012` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:44.003285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ea3d7882ee4c3dbf3f58082584e012)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:44.004327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING py.warnings [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:44.004327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: warnings.warn(msg) Jun 17 12:47:44.004327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: Jun 17 12:47:44.005522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:44.071723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ea3d7882ee4c3dbf3f58082584e012'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.091348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ea3d7882ee4c3dbf3f58082584e012'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.104004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ea3d7882ee4c3dbf3f58082584e012'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.112856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:47:44.121012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ea3d7882ee4c3dbf3f58082584e012'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.122264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ea3d7882ee4c3dbf3f58082584e012'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.129337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 754/1507] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:47:43 2022] DELETE /identity/v3/users/18ea3d7882ee4c3dbf3f58082584e012 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:47:44.140998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:44.141289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:44.141574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:44.159848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:47:44.161083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:47:44.228965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 754/1508] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:47:44.242369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:44.244529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:47:44.310433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc223ab-173b-4e84-bdf0-f48ca4e0b3b5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:44.314421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f1740-7529-416d-91da-829de1f79051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:44.315088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f1740-7529-416d-91da-829de1f79051 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:44.315388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f1740-7529-416d-91da-829de1f79051 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:44.315680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f1740-7529-416d-91da-829de1f79051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:44.326283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f1740-7529-416d-91da-829de1f79051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:44.327784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f1740-7529-416d-91da-829de1f79051 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:44.337857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 755/1509] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:47:44.705780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:47:44.707378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:47:44.719886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0910af3-41ad-4788-a57a-a68db4bc6dac None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:47:44.725183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:47:44.726106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:47:44.726390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:47:44.726675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] PATH_INFO: `/v3/projects/79bc762c6681485f9c84513470091d87` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:47:44.728027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bc762c6681485f9c84513470091d87)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:47:44.729385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:47:44.729385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:47:44.729385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:47:44.730899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:47:44.803644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bc762c6681485f9c84513470091d87', 'request_id': 'req-185c7a75-56a5-4d52-a434-16ec4d7cce72'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.818364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bc762c6681485f9c84513470091d87', 'request_id': 'req-185c7a75-56a5-4d52-a434-16ec4d7cce72'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.825932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bc762c6681485f9c84513470091d87', 'request_id': 'req-185c7a75-56a5-4d52-a434-16ec4d7cce72'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.842423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] The token cache is being invalidate because project 79bc762c6681485f9c84513470091d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:47:44.842724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.843396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-185c7a75-56a5-4d52-a434-16ec4d7cce72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:47:44.845155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 755/1510] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:47:44 2022] DELETE /identity/v3/projects/79bc762c6681485f9c84513470091d87 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:47:54.333318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=29 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:47:54.839421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:49:25.124152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d703-10bb-4f00-83d6-db548137233a None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:25.124962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d703-10bb-4f00-83d6-db548137233a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:25.125152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d703-10bb-4f00-83d6-db548137233a None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:25.127280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:49:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:25.135286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce5a2-a80e-4526-a002-177f4f397d96 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:25.136040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce5a2-a80e-4526-a002-177f4f397d96 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:25.136199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce5a2-a80e-4526-a002-177f4f397d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:25.156157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-45dce5a2-a80e-4526-a002-177f4f397d96 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:25.157387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-45dce5a2-a80e-4526-a002-177f4f397d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:25.218236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:25.395217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d703-10bb-4f00-83d6-db548137233a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:25.397954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b692d703-10bb-4f00-83d6-db548137233a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:25.478439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d703-10bb-4f00-83d6-db548137233a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:25.484122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac67e4-b0e7-4358-ac74-fa3b8fe6c574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:25.484751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac67e4-b0e7-4358-ac74-fa3b8fe6c574 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:25.485063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac67e4-b0e7-4358-ac74-fa3b8fe6c574 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:25.485339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac67e4-b0e7-4358-ac74-fa3b8fe6c574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:25.493655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac67e4-b0e7-4358-ac74-fa3b8fe6c574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:25.495256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac67e4-b0e7-4358-ac74-fa3b8fe6c574 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:25.504610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 757/1513] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:49:26.388573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ceae9-43e3-4fe8-8d74-8f8d9d69943a None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:26.389134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ceae9-43e3-4fe8-8d74-8f8d9d69943a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:26.389666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ceae9-43e3-4fe8-8d74-8f8d9d69943a None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:26.391062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:26.398198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782eee5-028c-40e8-af0e-167ae2b3c860 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:26.398681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782eee5-028c-40e8-af0e-167ae2b3c860 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:26.398789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782eee5-028c-40e8-af0e-167ae2b3c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:26.418867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1782eee5-028c-40e8-af0e-167ae2b3c860 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:26.420156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1782eee5-028c-40e8-af0e-167ae2b3c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:26.428648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:26.585167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ceae9-43e3-4fe8-8d74-8f8d9d69943a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:26.600934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ceae9-43e3-4fe8-8d74-8f8d9d69943a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:26.606853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce56abc-9844-4fae-aaed-e653a9bf7361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:26.607786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce56abc-9844-4fae-aaed-e653a9bf7361 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:26.608097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce56abc-9844-4fae-aaed-e653a9bf7361 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:26.608374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce56abc-9844-4fae-aaed-e653a9bf7361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:26.618198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce56abc-9844-4fae-aaed-e653a9bf7361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:26.619857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce56abc-9844-4fae-aaed-e653a9bf7361 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:26.629624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 758/1516] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:49:26.643502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4d623-d033-4820-80f0-874fe3bae823 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:26.643788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4d623-d033-4820-80f0-874fe3bae823 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:26.644064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4d623-d033-4820-80f0-874fe3bae823 None None] PATH_INFO: `/v3` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:26.645549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 759/1517] 10.222.0.43 () {54 vars in 925 bytes} [Fri Jun 17 12:49:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:49:26.652041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69bf89-3303-400b-b4d4-d5fefb86d1f4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:26.652334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69bf89-3303-400b-b4d4-d5fefb86d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:26.652617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69bf89-3303-400b-b4d4-d5fefb86d1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:26.670927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9a69bf89-3303-400b-b4d4-d5fefb86d1f4 None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:26.673028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9a69bf89-3303-400b-b4d4-d5fefb86d1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:26.741432 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 759/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 17 12:49:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:30.532177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77be897-7d7c-44ce-bf35-20fae7f413dd None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:30.532757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77be897-7d7c-44ce-bf35-20fae7f413dd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:30.532757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77be897-7d7c-44ce-bf35-20fae7f413dd None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:30.534127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 760/1519] 10.222.0.87 () {54 vars in 960 bytes} [Fri Jun 17 12:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:30.540224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803adc-cbc8-4ad2-a9ff-36cb96ba5c43 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:30.540686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803adc-cbc8-4ad2-a9ff-36cb96ba5c43 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:30.540800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803adc-cbc8-4ad2-a9ff-36cb96ba5c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:30.569841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-85803adc-cbc8-4ad2-a9ff-36cb96ba5c43 None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:30.571064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-85803adc-cbc8-4ad2-a9ff-36cb96ba5c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:30.582863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 760/1520] 10.222.0.87 () {58 vars in 1057 bytes} [Fri Jun 17 12:49:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:30.590820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77be897-7d7c-44ce-bf35-20fae7f413dd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:30.608153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77be897-7d7c-44ce-bf35-20fae7f413dd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:30.616197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b0b8-2074-412a-9ef5-b548e652c084 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:30.616876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b0b8-2074-412a-9ef5-b548e652c084 service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:30.617158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b0b8-2074-412a-9ef5-b548e652c084 service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:30.617424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b0b8-2074-412a-9ef5-b548e652c084 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:30.627525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b0b8-2074-412a-9ef5-b548e652c084 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:30.629139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b0b8-2074-412a-9ef5-b548e652c084 service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:30.639187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 761/1521] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:49:37.908930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6113f45-d891-44cc-8c7f-aded9e2b4728 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:37.909874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6113f45-d891-44cc-8c7f-aded9e2b4728 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:37.909874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6113f45-d891-44cc-8c7f-aded9e2b4728 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:37.912182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:49:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:37.919108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5c1ce-2a82-4be2-bbe1-19c4efadbdf7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:37.919678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5c1ce-2a82-4be2-bbe1-19c4efadbdf7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:37.919678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5c1ce-2a82-4be2-bbe1-19c4efadbdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:37.938994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9bc5c1ce-2a82-4be2-bbe1-19c4efadbdf7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:37.940254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9bc5c1ce-2a82-4be2-bbe1-19c4efadbdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:37.949302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:49:38.110880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6113f45-d891-44cc-8c7f-aded9e2b4728 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:38.132775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6113f45-d891-44cc-8c7f-aded9e2b4728 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:38.138627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce68a-73f1-4c9c-b328-c22eae392b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:38.139280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce68a-73f1-4c9c-b328-c22eae392b6f service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:38.139605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce68a-73f1-4c9c-b328-c22eae392b6f service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:38.139885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce68a-73f1-4c9c-b328-c22eae392b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:38.148362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce68a-73f1-4c9c-b328-c22eae392b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:38.150022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce68a-73f1-4c9c-b328-c22eae392b6f service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:38.159639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 762/1524] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:49:39.048497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce996c1f-e1b5-4662-88c1-67d57bd7ab46 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:39.049376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce996c1f-e1b5-4662-88c1-67d57bd7ab46 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:39.049376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce996c1f-e1b5-4662-88c1-67d57bd7ab46 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:39.052035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 12:49:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:39.059364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83fdaf-b16b-440d-a0de-44bbc5702766 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:39.059910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83fdaf-b16b-440d-a0de-44bbc5702766 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:39.059910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83fdaf-b16b-440d-a0de-44bbc5702766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:39.080747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4b83fdaf-b16b-440d-a0de-44bbc5702766 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:39.082338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4b83fdaf-b16b-440d-a0de-44bbc5702766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:39.092490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 12:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:49:39.247785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce996c1f-e1b5-4662-88c1-67d57bd7ab46 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:39.273344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce996c1f-e1b5-4662-88c1-67d57bd7ab46 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:39.279791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d5602-6075-4eaf-91db-25e3b989d3dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:39.280439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d5602-6075-4eaf-91db-25e3b989d3dd service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:39.280724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d5602-6075-4eaf-91db-25e3b989d3dd service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:39.281004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d5602-6075-4eaf-91db-25e3b989d3dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:39.290047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d5602-6075-4eaf-91db-25e3b989d3dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:39.291696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d5602-6075-4eaf-91db-25e3b989d3dd service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:39.301672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 764/1527] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:49:43.541307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943eb1-34b7-4660-9d85-6dff5a429596 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:43.542267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943eb1-34b7-4660-9d85-6dff5a429596 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:43.542267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943eb1-34b7-4660-9d85-6dff5a429596 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:43.544497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:49:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:49:43.553053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403a6b-db42-4b8c-a476-5c4433be9602 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:43.553724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403a6b-db42-4b8c-a476-5c4433be9602 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:43.553827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403a6b-db42-4b8c-a476-5c4433be9602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:43.573463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f1403a6b-db42-4b8c-a476-5c4433be9602 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:43.574685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f1403a6b-db42-4b8c-a476-5c4433be9602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:43.584568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:43.624379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943eb1-34b7-4660-9d85-6dff5a429596 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:43.626483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-92943eb1-34b7-4660-9d85-6dff5a429596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:43.687985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943eb1-34b7-4660-9d85-6dff5a429596 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:43.691722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e707c69-118f-44fb-aae7-a0561d84b599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:43.692370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707c69-118f-44fb-aae7-a0561d84b599 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:43.692651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707c69-118f-44fb-aae7-a0561d84b599 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:43.692926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707c69-118f-44fb-aae7-a0561d84b599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:43.702144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e707c69-118f-44fb-aae7-a0561d84b599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:43.703600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e707c69-118f-44fb-aae7-a0561d84b599 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:43.708877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:49:43.714196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 765/1530] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:49:45.806875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad42534-d98e-45a1-a6bb-dbb6b5927950 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:45.807860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad42534-d98e-45a1-a6bb-dbb6b5927950 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:45.807860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad42534-d98e-45a1-a6bb-dbb6b5927950 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:45.810214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:49:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:45.818651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac929f-917e-4305-9307-477201377d6c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:45.819155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac929f-917e-4305-9307-477201377d6c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:45.819264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac929f-917e-4305-9307-477201377d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:45.844182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c6ac929f-917e-4305-9307-477201377d6c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:45.845398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c6ac929f-917e-4305-9307-477201377d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:45.859671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:45.900470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad42534-d98e-45a1-a6bb-dbb6b5927950 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:45.902610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fad42534-d98e-45a1-a6bb-dbb6b5927950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:45.922219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad42534-d98e-45a1-a6bb-dbb6b5927950 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:45.926297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b982add-b39c-4c4d-8945-5a32e0cd90eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:45.927027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b982add-b39c-4c4d-8945-5a32e0cd90eb service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:45.927429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b982add-b39c-4c4d-8945-5a32e0cd90eb service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:45.927813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b982add-b39c-4c4d-8945-5a32e0cd90eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:45.937458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b982add-b39c-4c4d-8945-5a32e0cd90eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:45.938900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b982add-b39c-4c4d-8945-5a32e0cd90eb service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:45.948540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 767/1533] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:49:49.786478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dadb1a-bab7-4a2f-a998-5bc35c002ff1 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:49.787087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dadb1a-bab7-4a2f-a998-5bc35c002ff1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:49.787087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dadb1a-bab7-4a2f-a998-5bc35c002ff1 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:49.788441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:49.794753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4bed4-bd62-4420-a180-f3c818d77bc3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:49.795261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4bed4-bd62-4420-a180-f3c818d77bc3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:49.795653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4bed4-bd62-4420-a180-f3c818d77bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:49.822290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f8b4bed4-bd62-4420-a180-f3c818d77bc3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:49.823794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f8b4bed4-bd62-4420-a180-f3c818d77bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:49.833910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:49:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:50.114941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dadb1a-bab7-4a2f-a998-5bc35c002ff1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:50.117325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-49dadb1a-bab7-4a2f-a998-5bc35c002ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:50.197566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dadb1a-bab7-4a2f-a998-5bc35c002ff1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:50.201962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c952f3-97f3-49f9-bac9-ae6b3847cf8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:50.202641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c952f3-97f3-49f9-bac9-ae6b3847cf8d service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:50.202929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c952f3-97f3-49f9-bac9-ae6b3847cf8d service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:50.203358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c952f3-97f3-49f9-bac9-ae6b3847cf8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:50.214166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c952f3-97f3-49f9-bac9-ae6b3847cf8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:50.215665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c952f3-97f3-49f9-bac9-ae6b3847cf8d service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:50.225932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 768/1536] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:49:52.411984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a8345-c2a1-4379-b9c0-d832bf4b17eb None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:52.412523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a8345-c2a1-4379-b9c0-d832bf4b17eb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:52.412523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a8345-c2a1-4379-b9c0-d832bf4b17eb None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:52.413854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 12:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:49:52.420160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0aca1-1e77-4a47-99f9-4998e4cd2405 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:52.420646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0aca1-1e77-4a47-99f9-4998e4cd2405 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:52.420756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0aca1-1e77-4a47-99f9-4998e4cd2405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:52.444962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a9d0aca1-1e77-4a47-99f9-4998e4cd2405 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:49:52.446179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a9d0aca1-1e77-4a47-99f9-4998e4cd2405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:52.456299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 12:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:49:52.725020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a8345-c2a1-4379-b9c0-d832bf4b17eb None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:49:52.727147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-611a8345-c2a1-4379-b9c0-d832bf4b17eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:49:52.748525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a8345-c2a1-4379-b9c0-d832bf4b17eb None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:49:52.753300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7877794-5cc4-4954-964c-a8ee7f0352f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:49:52.753944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877794-5cc4-4954-964c-a8ee7f0352f4 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:49:52.754232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877794-5cc4-4954-964c-a8ee7f0352f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:49:52.754518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877794-5cc4-4954-964c-a8ee7f0352f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:49:52.764836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7877794-5cc4-4954-964c-a8ee7f0352f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:49:52.766284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7877794-5cc4-4954-964c-a8ee7f0352f4 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:49:52.778519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 770/1539] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:50:02.439209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=32 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:50:02.772365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=70 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:22.907289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76b895-3ef2-474a-9137-d417d7f8e673 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:22.907975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76b895-3ef2-474a-9137-d417d7f8e673 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:22.907975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76b895-3ef2-474a-9137-d417d7f8e673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:22.933565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ab76b895-3ef2-474a-9137-d417d7f8e673 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:22.935119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ab76b895-3ef2-474a-9137-d417d7f8e673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:22.950691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 770/1540] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:22.967752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330029-548f-4cef-80a8-a1352a18135a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:22.968393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330029-548f-4cef-80a8-a1352a18135a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:22.968393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330029-548f-4cef-80a8-a1352a18135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:22.999776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c4330029-548f-4cef-80a8-a1352a18135a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.000999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c4330029-548f-4cef-80a8-a1352a18135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.011617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 771/1541] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:51:23.017302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76b895-3ef2-474a-9137-d417d7f8e673 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.032997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76b895-3ef2-474a-9137-d417d7f8e673 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.041174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.041881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.042164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.043659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.043659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:23.043659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.045146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.063401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 771/1542] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:23.072350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4330029-548f-4cef-80a8-a1352a18135a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.082221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4330029-548f-4cef-80a8-a1352a18135a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.092221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6538-e612-4d68-be4d-3189d675e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.092913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6538-e612-4d68-be4d-3189d675e325 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.093193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6538-e612-4d68-be4d-3189d675e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.093469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6538-e612-4d68-be4d-3189d675e325 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.094189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6538-e612-4d68-be4d-3189d675e325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.096141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6538-e612-4d68-be4d-3189d675e325 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.144415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 772/1543] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:23.148573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.165055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8074f-2e4a-42e5-8f6a-0f679746a9b0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.247444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08c10d-705f-4db7-86a8-33ccc2d04050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.248146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08c10d-705f-4db7-86a8-33ccc2d04050 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.248427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08c10d-705f-4db7-86a8-33ccc2d04050 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.248701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08c10d-705f-4db7-86a8-33ccc2d04050 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.249421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08c10d-705f-4db7-86a8-33ccc2d04050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.251435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08c10d-705f-4db7-86a8-33ccc2d04050 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.259760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a727b-d60d-4588-bb9a-2fa4347d04cb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.260437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a727b-d60d-4588-bb9a-2fa4347d04cb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.260933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a727b-d60d-4588-bb9a-2fa4347d04cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.293164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a82a727b-d60d-4588-bb9a-2fa4347d04cb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.294788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a82a727b-d60d-4588-bb9a-2fa4347d04cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.301633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 772/1544] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:23.303014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bba66-df0d-418a-a750-b41cb9717068 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.303301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bba66-df0d-418a-a750-b41cb9717068 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.303675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bba66-df0d-418a-a750-b41cb9717068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.316187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 773/1545] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:51:23.318228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8066de-291a-4b9e-8b93-dece7319d931 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.318939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8066de-291a-4b9e-8b93-dece7319d931 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.319532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8066de-291a-4b9e-8b93-dece7319d931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.329705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ed8bba66-df0d-418a-a750-b41cb9717068 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.331175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ed8bba66-df0d-418a-a750-b41cb9717068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.358072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 773/1546] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.358312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ba8066de-291a-4b9e-8b93-dece7319d931 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.358529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824b8f-3eb1-4116-9fa8-5f66976f45f3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.358688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ba8066de-291a-4b9e-8b93-dece7319d931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.358844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824b8f-3eb1-4116-9fa8-5f66976f45f3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.358844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824b8f-3eb1-4116-9fa8-5f66976f45f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.374593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 774/1547] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.375555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8066de-291a-4b9e-8b93-dece7319d931 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.394338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8066de-291a-4b9e-8b93-dece7319d931 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.400284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e2824b8f-3eb1-4116-9fa8-5f66976f45f3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.402164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e2824b8f-3eb1-4116-9fa8-5f66976f45f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.407681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d700f3a-687e-4b23-9163-2d2b53fa098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.408377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d700f3a-687e-4b23-9163-2d2b53fa098d admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.408660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d700f3a-687e-4b23-9163-2d2b53fa098d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.408934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d700f3a-687e-4b23-9163-2d2b53fa098d admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.410381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d700f3a-687e-4b23-9163-2d2b53fa098d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.412478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d700f3a-687e-4b23-9163-2d2b53fa098d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.421551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 774/1548] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.422985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e6e6-3684-46d5-9cc1-eaadd0cc2fe7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.423279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e6e6-3684-46d5-9cc1-eaadd0cc2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.423707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e6e6-3684-46d5-9cc1-eaadd0cc2fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.436407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 775/1549] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:23.443095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61d79-4292-4831-98e8-a2b54f1a540c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.443465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61d79-4292-4831-98e8-a2b54f1a540c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.443765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61d79-4292-4831-98e8-a2b54f1a540c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.455636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ac46e6e6-3684-46d5-9cc1-eaadd0cc2fe7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.456877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ac46e6e6-3684-46d5-9cc1-eaadd0cc2fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.464243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a6c61d79-4292-4831-98e8-a2b54f1a540c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.465607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a6c61d79-4292-4831-98e8-a2b54f1a540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.467316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 775/1550] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.468874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec2044-4f41-446b-a2c3-bd3c91d1f73c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.469174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec2044-4f41-446b-a2c3-bd3c91d1f73c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.469517 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec2044-4f41-446b-a2c3-bd3c91d1f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.480827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 776/1551] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.482512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918de2-b278-4c38-8e1a-8962a5e13751 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.483012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918de2-b278-4c38-8e1a-8962a5e13751 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.483527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918de2-b278-4c38-8e1a-8962a5e13751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.489505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-6cec2044-4f41-446b-a2c3-bd3c91d1f73c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.490712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6cec2044-4f41-446b-a2c3-bd3c91d1f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.499445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 776/1552] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.500251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec2044-4f41-446b-a2c3-bd3c91d1f73c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.503962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-90918de2-b278-4c38-8e1a-8962a5e13751 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.505181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-90918de2-b278-4c38-8e1a-8962a5e13751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.512620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec2044-4f41-446b-a2c3-bd3c91d1f73c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.513040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 777/1553] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.514242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79c671-51de-41ca-84b7-7ad37cad2587 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.514524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79c671-51de-41ca-84b7-7ad37cad2587 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.514796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79c671-51de-41ca-84b7-7ad37cad2587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.517985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.518651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.518938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.519245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.520053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.521681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.532291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 777/1554] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:23.532581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-de79c671-51de-41ca-84b7-7ad37cad2587 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.533087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.533778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-de79c671-51de-41ca-84b7-7ad37cad2587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.542120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 778/1555] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.543331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211442-31ff-4f0e-8fd4-9feee0cbcca6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.543644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211442-31ff-4f0e-8fd4-9feee0cbcca6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.543875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e161e-1bf4-4df4-8373-f63be54f6c63 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.543995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211442-31ff-4f0e-8fd4-9feee0cbcca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.548376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.549033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.549323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.549608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.550349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:23.550659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.552323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.563285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-01211442-31ff-4f0e-8fd4-9feee0cbcca6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.564538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-01211442-31ff-4f0e-8fd4-9feee0cbcca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.565980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 778/1556] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:23.574013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.574187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 779/1557] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.575324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe1b5-dc96-429f-be32-a39f35808701 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.575624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe1b5-dc96-429f-be32-a39f35808701 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.575944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe1b5-dc96-429f-be32-a39f35808701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.579533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf239-6590-4ef2-a600-1c573568820b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.584305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.585032 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.585341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.586479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.586479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:23.586716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.588642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.597479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-56abe1b5-dc96-429f-be32-a39f35808701 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.598665 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-56abe1b5-dc96-429f-be32-a39f35808701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.601940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 779/1558] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:23.602827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.615464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 780/1559] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.615464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abe1b5-dc96-429f-be32-a39f35808701 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.615740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69500748-cfe6-4d18-b130-8ec43c3e73b8 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.618520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abe1b5-dc96-429f-be32-a39f35808701 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.620816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.621510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.621798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.622110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.622848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:23.623162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.623859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.624527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.624812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.624925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.625086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.625814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:23.626159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.627225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING py.warnings [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:51:23.627225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: warnings.warn(msg) Jun 17 12:51:23.627225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: Jun 17 12:51:23.628297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58f565-602d-478d-aa10-581dfe8d32f7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.642769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 780/1560] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:23.642769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.643021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 781/1561] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:23.645157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc34f-c1a0-4a5a-b676-58edaf146010 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.645157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc34f-c1a0-4a5a-b676-58edaf146010 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.645157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc34f-c1a0-4a5a-b676-58edaf146010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.647959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0bfed-ecd8-4f04-89fc-8a96890ef0d0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.652511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98039653-d5ac-4072-8b6a-7b80e2df44dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.653409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039653-d5ac-4072-8b6a-7b80e2df44dd admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.653691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039653-d5ac-4072-8b6a-7b80e2df44dd admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.653966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039653-d5ac-4072-8b6a-7b80e2df44dd admin admin] PATH_INFO: `/v3/projects/f796e354ac1d43318d0c3abc7a7201ae/users/52c698db84684181b1e4247f72c5b2d5/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.672796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-961fc34f-c1a0-4a5a-b676-58edaf146010 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.673859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98039653-d5ac-4072-8b6a-7b80e2df44dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f796e354ac1d43318d0c3abc7a7201ae, user_id=52c698db84684181b1e4247f72c5b2d5, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.674045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-961fc34f-c1a0-4a5a-b676-58edaf146010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.674932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-98039653-d5ac-4072-8b6a-7b80e2df44dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:51:23.674932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:51:23.674932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:51:23.676994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98039653-d5ac-4072-8b6a-7b80e2df44dd admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.683828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 782/1562] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.684917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fc34f-c1a0-4a5a-b676-58edaf146010 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.692594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 781/1563] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:23 2022] PUT /identity/v3/projects/f796e354ac1d43318d0c3abc7a7201ae/users/52c698db84684181b1e4247f72c5b2d5/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:23.694092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d26373-ab9b-4833-ac9d-cd9d3b523a3a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.694476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d26373-ab9b-4833-ac9d-cd9d3b523a3a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.694759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d26373-ab9b-4833-ac9d-cd9d3b523a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.705329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fc34f-c1a0-4a5a-b676-58edaf146010 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.713452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a6d26373-ab9b-4833-ac9d-cd9d3b523a3a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.714695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a6d26373-ab9b-4833-ac9d-cd9d3b523a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.740177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:23.761832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e52795-6f3c-4b75-8fff-ed0a8aa619e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.762509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52795-6f3c-4b75-8fff-ed0a8aa619e8 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.762790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52795-6f3c-4b75-8fff-ed0a8aa619e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.763064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52795-6f3c-4b75-8fff-ed0a8aa619e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.763802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e52795-6f3c-4b75-8fff-ed0a8aa619e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.765718 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e52795-6f3c-4b75-8fff-ed0a8aa619e8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.776010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 782/1564] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.776886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d26373-ab9b-4833-ac9d-cd9d3b523a3a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.786413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d26373-ab9b-4833-ac9d-cd9d3b523a3a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.791720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.792383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.792659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.792931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.793619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.794646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:51:23.794646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:51:23.794646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:51:23.795851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.799697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 783/1565] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:23.799697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589666e-c1ed-4fe0-a25a-98372d1a29a8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.799697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589666e-c1ed-4fe0-a25a-98372d1a29a8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.799697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589666e-c1ed-4fe0-a25a-98372d1a29a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.818177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a589666e-c1ed-4fe0-a25a-98372d1a29a8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.819775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a589666e-c1ed-4fe0-a25a-98372d1a29a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.829322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 783/1566] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:23.829322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.854873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:23.881408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 784/1567] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:23.882371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589666e-c1ed-4fe0-a25a-98372d1a29a8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.892398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589666e-c1ed-4fe0-a25a-98372d1a29a8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.897366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.898022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.898488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.898769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.899151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbbebb-75a7-43fb-9e02-4fcb7b3625e4 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:23.899534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.901451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.903721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1167f8-7abf-4fb8-8a4c-115f391332d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:23.904382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1167f8-7abf-4fb8-8a4c-115f391332d1 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.904666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1167f8-7abf-4fb8-8a4c-115f391332d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.904937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1167f8-7abf-4fb8-8a4c-115f391332d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.905627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1167f8-7abf-4fb8-8a4c-115f391332d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:23.907543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1167f8-7abf-4fb8-8a4c-115f391332d1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:23.927775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 785/1568] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:23.928428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:23.937143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 784/1569] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:23.937143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04f691-975f-41aa-9c41-076a4363f5b4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:23.937143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04f691-975f-41aa-9c41-076a4363f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:23.937143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04f691-975f-41aa-9c41-076a4363f5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:23.970271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4a04f691-975f-41aa-9c41-076a4363f5b4 None None] MFA Rules not processed for user `52c698db84684181b1e4247f72c5b2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:23.971683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4a04f691-975f-41aa-9c41-076a4363f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:23.995398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9838-0cae-4c67-80b4-82d81993bd4b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.035579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 785/1570] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:24.036525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04f691-975f-41aa-9c41-076a4363f5b4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.059165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.059887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.060170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.060448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.061196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:24.061509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.062155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:24.063701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.082989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 786/1571] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:24.082989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.086434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d445f-81ad-4bce-9730-b4e70122640c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.092065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.092722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.093007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.093309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.093977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.094986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING py.warnings [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:51:24.094986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: warnings.warn(msg) Jun 17 12:51:24.094986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: Jun 17 12:51:24.096219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.108701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04f691-975f-41aa-9c41-076a4363f5b4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.116968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.116968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.116968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.116968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.117474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:24.117474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.118650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.121534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 787/1572] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.122404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.136034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 786/1573] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:24.136252 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32f50-8ce8-431b-80a0-880dfafaf85e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.136351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.139930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.140602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.140864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.141161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.141853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.143818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.145850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d9f36-017f-49ab-8383-ccc938b2e6f6 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.151117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.151795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.152077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.152351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.153074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:24.153378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.155140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.164214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 788/1574] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.168198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.168346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 787/1575] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:24.168985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.182172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30768da-4eb2-45bb-a88c-246cdc856757 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.183038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1ceae-d6ae-4da8-a6ac-9f8bff2ec2b7 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.187684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.188158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.188341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.188605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.188856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.189010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.189145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.189449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.189601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.190139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.191461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.192181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.213753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 789/1576] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.214606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.215711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 788/1577] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.216501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.226720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c43a9c-03ff-42a0-96e3-6479b68a6321 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.228777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fb069-a02a-48af-ae9d-13d40cf7261a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.231865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.232712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.232999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.233271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.233438 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.233985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.234106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.234365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.234639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.235325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.235932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.237254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.239252 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:24.265581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 790/1578] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.266672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.266829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 789/1579] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.267741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.331445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453c538-6089-42fe-b3e2-0b1643a1cd60 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.332978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcfadd-26bc-494d-ad29-69dfe968b864 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.336901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.337616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.337748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.338256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.338520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.338671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.338956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.339223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.339319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.340121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.341266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.341720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.351989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 791/1580] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.352819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.362077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb05b1-c134-453d-8d04-6d4fb09a382c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.366948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.367639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.367921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.368197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.368920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.370507 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.375020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 790/1581] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.375020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.380858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 792/1582] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.381528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.393403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e47d18-b01d-4ba6-b1bd-2e8a07e522c6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.427423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:24.442921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd2634-a8bf-46e3-9ce4-7775f49184f3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.447787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.448464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.448748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.449022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.449773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.450847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.451552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.451735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.451863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.452099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.452839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.454469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.465116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 793/1583] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.466039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.475751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800e0f9-8fbb-4c55-9b4b-4b623277698a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.480661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 791/1584] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.480831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.481317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.481599 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.481879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.482638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.484282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.484425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.496228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 794/1585] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.497512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.508830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4f2cc-f363-4524-8453-5e9d8edfa3db admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.549991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336457c-a881-4421-984b-66bfe7f421f4 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.554598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.555287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.555751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.556033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.556733 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.558809 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.560047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:24.567948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.568642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.569387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.569739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.570544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.572692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.579976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 792/1586] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.580876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.593215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee2e4f-0ae3-4836-8361-3ce860c79a67 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.594865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 795/1587] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.595782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.598018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.598678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.598969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.599249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.600040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.601760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.606402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1089e-7ebd-4f2d-bee4-d5af9857d951 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.611903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.612378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 793/1588] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.612549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.612838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.613091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.613194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.613765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.615741 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.625603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b818d14-fec7-4429-ac39-18c5897c5f34 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.630918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.631643 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.631928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.632226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.632953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.634548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.636000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 796/1589] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.636925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.646163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161350-39b2-4a2d-a053-104ec588c757 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.649172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 794/1590] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.650000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.654459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.655123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.655430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.655712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.656448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.658110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.661661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06db46e-d813-49a0-8011-ef2331832001 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.666468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.667124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.667432 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.667713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.668398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.669130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 797/1591] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.669994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.670364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.679570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b2dd0-ecb3-46e8-b702-c6c60921a90e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.684885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.685628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.685916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.686198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.686916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.688518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.693818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 795/1592] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:24 2022] POST /identity/v3/users => generated 284 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:24.693818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.698906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 798/1593] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.699814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.705680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cc4ec-2f59-47b1-a19c-9bcac6ac84b3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.709467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b1ac8-8dc2-429c-8d0d-78eaf42cd9be admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.710512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.711254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.711640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.711991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.712797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.714727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.714986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.715390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.715677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.715955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.716677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.718271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.725174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 796/1594] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.726255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.729129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 799/1595] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.729951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.739027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f94de2-1405-4d53-81f7-23140b83dd2f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.739386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b151f-d829-49fd-956b-09e81db188c2 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.743751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.743929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.744570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.744695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.744853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.744950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.745129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.745241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] PATH_INFO: `/v3/projects/ea1bbb589db14084aff24be5d6e118a3/users/9863e406790b413faf022554ba4b8dfe/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.745850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.747438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.757793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 800/1596] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.758670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.759989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea1bbb589db14084aff24be5d6e118a3, user_id=9863e406790b413faf022554ba4b8dfe, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.763439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.768500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294d676-fb32-42a4-abd6-d2d544a6d238 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.774432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.775399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.775808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.775965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] PATH_INFO: `/v3/projects/8ba886227347427bb1cbce939509be05/users/efee42b434cd47d99984b85f677983c2/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.778857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 797/1597] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:24 2022] PUT /identity/v3/projects/ea1bbb589db14084aff24be5d6e118a3/users/9863e406790b413faf022554ba4b8dfe/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:24.782698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.792059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba886227347427bb1cbce939509be05, user_id=efee42b434cd47d99984b85f677983c2, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.794745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.808097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 801/1598] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:24 2022] PUT /identity/v3/projects/8ba886227347427bb1cbce939509be05/users/efee42b434cd47d99984b85f677983c2/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:24.811872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.831159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:24.846082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08a426-a8fb-472f-9604-97df531beec1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.871744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e74525-cd04-4c61-971e-15b5b29d37bf admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.876629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.877309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.877595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.877873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.878618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.880299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.890827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 802/1599] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:24.891655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.901738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd789c-ac4b-4eff-88bf-a5355a53cc5b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:24.904093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.905042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.905327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.905609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] PATH_INFO: `/v3/projects/ead9b99d062e49a88a88e8b37a3b3519/users/f3bfb04b34ed4761bb09be7b29d1ec8f/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.906670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:24.907572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:24.907857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:24.908130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] PATH_INFO: `/v3/projects/652fca4387ef47078860b964334a8651/users/a17b8a752d78431dbde11be6d0812de3/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:24.920010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead9b99d062e49a88a88e8b37a3b3519, user_id=f3bfb04b34ed4761bb09be7b29d1ec8f, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.922725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.923111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652fca4387ef47078860b964334a8651, user_id=a17b8a752d78431dbde11be6d0812de3, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:24.925785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:24.936865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 798/1600] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:24 2022] PUT /identity/v3/projects/ead9b99d062e49a88a88e8b37a3b3519/users/f3bfb04b34ed4761bb09be7b29d1ec8f/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:24.937529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.942558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 803/1601] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:24 2022] PUT /identity/v3/projects/652fca4387ef47078860b964334a8651/users/a17b8a752d78431dbde11be6d0812de3/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:24.943545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:24.978553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:25.002322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fecfb4-a4da-4687-9267-4e04693663c0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.007099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298bd98-1794-44ac-a96e-1316a88effd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.008152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298bd98-1794-44ac-a96e-1316a88effd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.008441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298bd98-1794-44ac-a96e-1316a88effd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.008723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298bd98-1794-44ac-a96e-1316a88effd6 admin admin] PATH_INFO: `/v3/projects/78332df60e584f23ac09528633d288d0/users/dc954e7b8a2c481d9da4b29b7a05f3e9/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.009159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee637a-683e-406b-ac04-ce0a4875edd2 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.013867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.014536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.014822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.015099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.015879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.017494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.023943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298bd98-1794-44ac-a96e-1316a88effd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78332df60e584f23ac09528633d288d0, user_id=dc954e7b8a2c481d9da4b29b7a05f3e9, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.027002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298bd98-1794-44ac-a96e-1316a88effd6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.030740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 804/1602] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:25.030740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.038332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5f27c-b889-4861-b50a-db2da7e9afdd admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.042803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 799/1603] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:24 2022] PUT /identity/v3/projects/78332df60e584f23ac09528633d288d0/users/dc954e7b8a2c481d9da4b29b7a05f3e9/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:25.044805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1c35-54df-43a5-b992-4f82fe861921 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.045192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1c35-54df-43a5-b992-4f82fe861921 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.045583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1c35-54df-43a5-b992-4f82fe861921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.062229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-2a4e1c35-54df-43a5-b992-4f82fe861921 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.063490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2a4e1c35-54df-43a5-b992-4f82fe861921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.096500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b447-a8e2-409e-b7cf-c224833a05af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.097266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b447-a8e2-409e-b7cf-c224833a05af admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.097552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b447-a8e2-409e-b7cf-c224833a05af admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.097828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b447-a8e2-409e-b7cf-c224833a05af admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.098591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b447-a8e2-409e-b7cf-c224833a05af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.100264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b447-a8e2-409e-b7cf-c224833a05af admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.113440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 805/1604] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:25.113440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf92bd-8503-44a8-885a-e16c2d9615c7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.113440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf92bd-8503-44a8-885a-e16c2d9615c7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.113440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf92bd-8503-44a8-885a-e16c2d9615c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.129479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 800/1605] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.130455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e1c35-54df-43a5-b992-4f82fe861921 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.134072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9abf92bd-8503-44a8-885a-e16c2d9615c7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.135421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9abf92bd-8503-44a8-885a-e16c2d9615c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.137544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:25.143396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e1c35-54df-43a5-b992-4f82fe861921 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.148587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 806/1606] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.148587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094b6ae-0b27-4ccf-aaa9-76eb05747e1b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.148587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094b6ae-0b27-4ccf-aaa9-76eb05747e1b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.148587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094b6ae-0b27-4ccf-aaa9-76eb05747e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.152418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5cc3-6d74-439a-b819-92271302d392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.153192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cc3-6d74-439a-b819-92271302d392 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.153549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cc3-6d74-439a-b819-92271302d392 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.153894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cc3-6d74-439a-b819-92271302d392 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.154705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d5cc3-6d74-439a-b819-92271302d392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.156376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d5cc3-6d74-439a-b819-92271302d392 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.166275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-2094b6ae-0b27-4ccf-aaa9-76eb05747e1b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.166821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 801/1607] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:25.170083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2094b6ae-0b27-4ccf-aaa9-76eb05747e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.170201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f894a07-f426-4981-a523-eeb1a11a075d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.170201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f894a07-f426-4981-a523-eeb1a11a075d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.170201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f894a07-f426-4981-a523-eeb1a11a075d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.175864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 807/1608] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.176678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094b6ae-0b27-4ccf-aaa9-76eb05747e1b None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.186573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094b6ae-0b27-4ccf-aaa9-76eb05747e1b None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.187085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8f894a07-f426-4981-a523-eeb1a11a075d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.188358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8f894a07-f426-4981-a523-eeb1a11a075d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.191221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.192150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.192460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.192739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] PATH_INFO: `/v3/projects/802bb24d4c7c4040bd844a7b84698a71/users/77eced3e20b3465d932e594b7149301c/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.196025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 802/1609] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.196806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f894a07-f426-4981-a523-eeb1a11a075d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.206338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f894a07-f426-4981-a523-eeb1a11a075d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.210954 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.211654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.211942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.212221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.213305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.213474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=802bb24d4c7c4040bd844a7b84698a71, user_id=77eced3e20b3465d932e594b7149301c, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.214927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.216198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.226470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 803/1610] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:25.227593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.230813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 808/1611] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:25 2022] PUT /identity/v3/projects/802bb24d4c7c4040bd844a7b84698a71/users/77eced3e20b3465d932e594b7149301c/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:25.231494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.238366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff3592-fed9-422d-b400-200ae5e98d9e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.292114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa95e0-9fe3-4573-89b6-7098fcd2ad6d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.295711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.296536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.296818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.297092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.297980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.298890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.299173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.299487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] PATH_INFO: `/v3/projects/b3495f3daaa44dc9b20a08f3e1128bfa/users/a14c1230233e43789087e9239ffb91e2/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.306209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.307685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.315161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3495f3daaa44dc9b20a08f3e1128bfa, user_id=a14c1230233e43789087e9239ffb91e2, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.317275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 809/1612] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:25.317873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.318064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.326012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8481ab-2987-42a0-a5d2-79a5c64ad8a6 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.334259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 804/1613] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:25 2022] PUT /identity/v3/projects/b3495f3daaa44dc9b20a08f3e1128bfa/users/a14c1230233e43789087e9239ffb91e2/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:25.335287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.361328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:25.389428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.390115 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.390412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.390678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.412756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa552b-e0db-4906-a870-0bc1b4dd0ed7 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.418482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.419417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.419705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.419978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] PATH_INFO: `/v3/projects/00745c24508d4170878017dcc2e0294b/users/7392dcf303da4771b6a40ab48dcffcd0/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.437920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00745c24508d4170878017dcc2e0294b, user_id=7392dcf303da4771b6a40ab48dcffcd0, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.440955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.457966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 805/1614] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:25 2022] PUT /identity/v3/projects/00745c24508d4170878017dcc2e0294b/users/7392dcf303da4771b6a40ab48dcffcd0/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:25.458911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.471683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.473438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.484560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 810/1615] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:25.485371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.494640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea339d6-65d6-4fdd-97a2-8abf4da9063d service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.510788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:25.529320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a5e7-a32d-4bf3-a1f4-785fcfd8e067 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.533512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.534180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.534477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.534881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.553031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14deed7-5581-40b3-8edd-076ad91ce5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.553792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14deed7-5581-40b3-8edd-076ad91ce5ea service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.554117 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14deed7-5581-40b3-8edd-076ad91ce5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.554958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14deed7-5581-40b3-8edd-076ad91ce5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.604738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.606372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.617845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 806/1616] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:25.618709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.622297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14deed7-5581-40b3-8edd-076ad91ce5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.623941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14deed7-5581-40b3-8edd-076ad91ce5ea service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.629234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d706bb6-d9a2-4e95-8719-d6a5367fdc4f service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.634700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156d89f-510a-4f93-81cd-79604038a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:25.635468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 811/1617] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:25.635867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d89f-510a-4f93-81cd-79604038a669 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.636162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d89f-510a-4f93-81cd-79604038a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.636446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d89f-510a-4f93-81cd-79604038a669 admin admin] PATH_INFO: `/v3/projects/78332df60e584f23ac09528633d288d0/users/dc954e7b8a2c481d9da4b29b7a05f3e9/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.637044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126654a9-2eb5-44a6-ae8c-142f493745ab None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.637447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126654a9-2eb5-44a6-ae8c-142f493745ab None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.637812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126654a9-2eb5-44a6-ae8c-142f493745ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.638480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156d89f-510a-4f93-81cd-79604038a669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78332df60e584f23ac09528633d288d0, user_id=dc954e7b8a2c481d9da4b29b7a05f3e9, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:25.641908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156d89f-510a-4f93-81cd-79604038a669 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:25.666265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 807/1618] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:25 2022] PUT /identity/v3/projects/78332df60e584f23ac09528633d288d0/users/dc954e7b8a2c481d9da4b29b7a05f3e9/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:25.667352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-126654a9-2eb5-44a6-ae8c-142f493745ab None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.667995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afeeb-7459-48ff-9d40-523babef83b0 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.668447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afeeb-7459-48ff-9d40-523babef83b0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.668871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afeeb-7459-48ff-9d40-523babef83b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.668983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-126654a9-2eb5-44a6-ae8c-142f493745ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.699808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:25.708783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c29afeeb-7459-48ff-9d40-523babef83b0 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.710157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c29afeeb-7459-48ff-9d40-523babef83b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.771344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 812/1619] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.771344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee8445-3971-4690-8879-2f53e73f869c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.771344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee8445-3971-4690-8879-2f53e73f869c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.771344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee8445-3971-4690-8879-2f53e73f869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.803395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-28ee8445-3971-4690-8879-2f53e73f869c None None] MFA Rules not processed for user `7392dcf303da4771b6a40ab48dcffcd0`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.804923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-28ee8445-3971-4690-8879-2f53e73f869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.819751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 808/1620] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.821243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fcde3-bc08-426b-90f2-9774cbe9e64f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:25.821538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fcde3-bc08-426b-90f2-9774cbe9e64f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:25.821819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fcde3-bc08-426b-90f2-9774cbe9e64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:25.864971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-651fcde3-bc08-426b-90f2-9774cbe9e64f None None] MFA Rules not processed for user `dc954e7b8a2c481d9da4b29b7a05f3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:25.867769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-651fcde3-bc08-426b-90f2-9774cbe9e64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:25.908686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 813/1621] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.909813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee8445-3971-4690-8879-2f53e73f869c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.983011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 809/1622] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:25.984350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fcde3-bc08-426b-90f2-9774cbe9e64f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:25.995216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee8445-3971-4690-8879-2f53e73f869c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:25.998229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:26.000421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:26.001097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:26.001383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:26.001768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:26.013962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:26.015491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:26.028191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 814/1623] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:26.029337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:26.071481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fcde3-bc08-426b-90f2-9774cbe9e64f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:26.077386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:26.078139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:26.078465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:26.078779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:26.093138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386e04-3658-4533-9049-287d86b33d86 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:26.094741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:26.096268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:26.097153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:26.097490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:26.097780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:26.098060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:26.107723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:26.109425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:26.112012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 810/1624] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:26.112976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:26.134388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d4701-981f-4d67-91f3-1abf93c6058b service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:26.139242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 815/1625] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:26.144557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c593-e610-487c-a22d-3d871dbee2f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:26.145251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c593-e610-487c-a22d-3d871dbee2f7 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:26.145558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c593-e610-487c-a22d-3d871dbee2f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:26.145837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c593-e610-487c-a22d-3d871dbee2f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:26.158940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c593-e610-487c-a22d-3d871dbee2f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:26.160663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c593-e610-487c-a22d-3d871dbee2f7 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:26.174012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 811/1626] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:26.319691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:26.357931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:26.373331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1fffa-f4a5-4197-ae3a-d1a4f8d6cd04 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:26.387738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600d38b-6164-43e3-a100-2ec4816cfec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:26.390730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600d38b-6164-43e3-a100-2ec4816cfec3 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:26.391079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600d38b-6164-43e3-a100-2ec4816cfec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:26.392653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600d38b-6164-43e3-a100-2ec4816cfec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:26.604933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600d38b-6164-43e3-a100-2ec4816cfec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:26.609776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600d38b-6164-43e3-a100-2ec4816cfec3 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:26.659869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 816/1627] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:28.587879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46288fc4-9bb4-45bf-bed0-7a8cc97f1333 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:28.588800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46288fc4-9bb4-45bf-bed0-7a8cc97f1333 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:28.588800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46288fc4-9bb4-45bf-bed0-7a8cc97f1333 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:28.590171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 812/1628] 10.222.0.87 () {54 vars in 961 bytes} [Fri Jun 17 12:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:51:28.600054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aec7b-9829-4375-9f41-44a86ae029cd None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:28.600715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aec7b-9829-4375-9f41-44a86ae029cd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:28.600715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aec7b-9829-4375-9f41-44a86ae029cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:28.653473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8b1aec7b-9829-4375-9f41-44a86ae029cd None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:28.654749 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8b1aec7b-9829-4375-9f41-44a86ae029cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:28.776691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 817/1629] 10.222.0.87 () {58 vars in 1058 bytes} [Fri Jun 17 12:51:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:28.785020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34866427-6fd0-48ef-965d-cb4343431144 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:28.785388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34866427-6fd0-48ef-965d-cb4343431144 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:28.785733 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34866427-6fd0-48ef-965d-cb4343431144 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:28.787753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 813/1630] 10.222.0.87 () {54 vars in 912 bytes} [Fri Jun 17 12:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:51:28.797284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97e2cd-34b8-4d2a-b346-d936d134611b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:28.797668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97e2cd-34b8-4d2a-b346-d936d134611b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:28.798010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97e2cd-34b8-4d2a-b346-d936d134611b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:28.840137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1f97e2cd-34b8-4d2a-b346-d936d134611b None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:28.841718 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1f97e2cd-34b8-4d2a-b346-d936d134611b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:28.869062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 818/1631] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:51:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:28.883693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34866427-6fd0-48ef-965d-cb4343431144 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:28.902327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34866427-6fd0-48ef-965d-cb4343431144 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:28.915856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:28.916810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:28.917523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:28.917966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:28.936221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:28.940039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:28.964669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 814/1632] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:30.730256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1fef2-8adb-454b-8fd3-e6fda92bfd20 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:30.733112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1fef2-8adb-454b-8fd3-e6fda92bfd20 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:30.735075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1fef2-8adb-454b-8fd3-e6fda92bfd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:30.801299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8da1fef2-8adb-454b-8fd3-e6fda92bfd20 None None] MFA Rules not processed for user `77eced3e20b3465d932e594b7149301c`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:30.802946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8da1fef2-8adb-454b-8fd3-e6fda92bfd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:30.923809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 819/1633] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:30.925158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:30.938654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118a3a-3e98-45ce-9655-2f3c4dd74d68 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:30.953624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:30.954455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:30.954839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:30.955237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:30.956093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:30.958144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:31.034582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 815/1634] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:30 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:31.038290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1fef2-8adb-454b-8fd3-e6fda92bfd20 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:31.064573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1fef2-8adb-454b-8fd3-e6fda92bfd20 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:31.104640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:31.202590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:31.203471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.203757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.204035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.204745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:31.206700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:31.252479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 820/1635] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:31 2022] POST /identity/v3/users => generated 309 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:31.256952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:31.273508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514561d9-192e-4fc8-8ec2-28dfe477406d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:31.290349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145be85f-7a04-443c-bd4c-85651581fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:31.293926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be85f-7a04-443c-bd4c-85651581fd36 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.294761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be85f-7a04-443c-bd4c-85651581fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.295849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be85f-7a04-443c-bd4c-85651581fd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.307880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145be85f-7a04-443c-bd4c-85651581fd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:31.315455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145be85f-7a04-443c-bd4c-85651581fd36 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:31.369599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 816/1636] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:31.370145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:31.377516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab0ff-197b-4bdd-a72f-b08d205c94c3 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.378532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab0ff-197b-4bdd-a72f-b08d205c94c3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.378864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab0ff-197b-4bdd-a72f-b08d205c94c3 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.385771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf730b93-a1c8-431d-b0c7-64ac6214a7f9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:31.387441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 817/1637] 10.222.0.87 () {54 vars in 961 bytes} [Fri Jun 17 12:51:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:51:31.399584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:31.401434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497b6c6-c6d0-4ad1-a682-13f47d9e9989 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.401672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.401968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.402077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497b6c6-c6d0-4ad1-a682-13f47d9e9989 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.402301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] PATH_INFO: `/v3/projects/26dc5562058a400c8965f2739493d3e8/users/ba13730fb72b4db1bbd531cd5c565e3a/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.402405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497b6c6-c6d0-4ad1-a682-13f47d9e9989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.440727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] RBAC: Authorizing `identity:create_grant(project_id=26dc5562058a400c8965f2739493d3e8, user_id=ba13730fb72b4db1bbd531cd5c565e3a, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:31.443445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:31.469856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b497b6c6-c6d0-4ad1-a682-13f47d9e9989 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:31.473692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b497b6c6-c6d0-4ad1-a682-13f47d9e9989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:31.482464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 821/1638] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:31 2022] PUT /identity/v3/projects/26dc5562058a400c8965f2739493d3e8/users/ba13730fb72b4db1bbd531cd5c565e3a/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:31.483240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:31.644750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c9b42-ab89-4278-9b03-f8f367ba0def admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:31.653210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:31.653908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.654194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.654463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.655216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:31.656861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:31.684174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 822/1639] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:31.685613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:31.714422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41c6d8-a38e-4771-bfbb-68e6c4ca0f06 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:31.720254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:31.721177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.721460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.721787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] PATH_INFO: `/v3/projects/26dc5562058a400c8965f2739493d3e8/users/ba13730fb72b4db1bbd531cd5c565e3a/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.727890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=26dc5562058a400c8965f2739493d3e8, user_id=ba13730fb72b4db1bbd531cd5c565e3a, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:31.730554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:31.733395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 818/1640] 10.222.0.87 () {58 vars in 1058 bytes} [Fri Jun 17 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:51:31.739313 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbf515-0c74-452b-8a75-0550175cb310 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.739726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbf515-0c74-452b-8a75-0550175cb310 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.740006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbf515-0c74-452b-8a75-0550175cb310 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.743533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 819/1641] 10.222.0.87 () {54 vars in 912 bytes} [Fri Jun 17 12:51:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:51:31.749164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91b228-c8ad-419a-b606-54566a975f65 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:31.749452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91b228-c8ad-419a-b606-54566a975f65 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:31.749768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91b228-c8ad-419a-b606-54566a975f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:31.761502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 823/1642] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:31 2022] PUT /identity/v3/projects/26dc5562058a400c8965f2739493d3e8/users/ba13730fb72b4db1bbd531cd5c565e3a/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:31.799461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ad91b228-c8ad-419a-b606-54566a975f65 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:31.800706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ad91b228-c8ad-419a-b606-54566a975f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:31.902044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:32.001600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 820/1643] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:32.016538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:32.166466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aaa0d-bdcb-4996-b453-7eaebed3c9bf admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:32.166466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab8631-9716-4d8d-b5f5-eea08ee8465e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:32.166466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8631-9716-4d8d-b5f5-eea08ee8465e service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:32.166466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8631-9716-4d8d-b5f5-eea08ee8465e service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:32.166842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8631-9716-4d8d-b5f5-eea08ee8465e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:32.179536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab8631-9716-4d8d-b5f5-eea08ee8465e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:32.181401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab8631-9716-4d8d-b5f5-eea08ee8465e service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:32.214330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 824/1644] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:35.738415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0878c-2fa5-46cf-a7a8-70b392dc9d42 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:35.738939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0878c-2fa5-46cf-a7a8-70b392dc9d42 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:35.738939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0878c-2fa5-46cf-a7a8-70b392dc9d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:35.808554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-cbf0878c-2fa5-46cf-a7a8-70b392dc9d42 None None] MFA Rules not processed for user `ba13730fb72b4db1bbd531cd5c565e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:35.812077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cbf0878c-2fa5-46cf-a7a8-70b392dc9d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:35.970163 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 821/1645] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:35 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:39.612894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8cbfc-f13f-4cb2-8ed5-d5bd10017afc None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:39.613916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8cbfc-f13f-4cb2-8ed5-d5bd10017afc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:39.614433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8cbfc-f13f-4cb2-8ed5-d5bd10017afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:39.654525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:39.678926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7eb8cbfc-f13f-4cb2-8ed5-d5bd10017afc None None] MFA Rules not processed for user `9863e406790b413faf022554ba4b8dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:39.680388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7eb8cbfc-f13f-4cb2-8ed5-d5bd10017afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:39.777553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8b45-b8c1-497c-b08d-bcccc665fa47 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:39.778159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8b45-b8c1-497c-b08d-bcccc665fa47 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:39.778159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8b45-b8c1-497c-b08d-bcccc665fa47 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:39.779786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 822/1646] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:51:39.864681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b8b45-b8c1-497c-b08d-bcccc665fa47 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:39.892287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 825/1647] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:39.912383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8cbfc-f13f-4cb2-8ed5-d5bd10017afc None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:39.930495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8cbfc-f13f-4cb2-8ed5-d5bd10017afc None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:39.944442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6754195-4e77-474d-9101-91d26db5c068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:39.945219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754195-4e77-474d-9101-91d26db5c068 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:39.945817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754195-4e77-474d-9101-91d26db5c068 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:39.946350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754195-4e77-474d-9101-91d26db5c068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:39.969397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6754195-4e77-474d-9101-91d26db5c068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:39.983359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6754195-4e77-474d-9101-91d26db5c068 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:40.043832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 826/1648] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:40.152723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b8b45-b8c1-497c-b08d-bcccc665fa47 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:40.168068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:40.169588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:40.171056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:40.171706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] PATH_INFO: `/v3/users/52c698db84684181b1e4247f72c5b2d5` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:40.179334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c698db84684181b1e4247f72c5b2d5)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:40.180471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: WARNING py.warnings [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:51:40.180471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: warnings.warn(msg) Jun 17 12:51:40.180471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: Jun 17 12:51:40.181711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:40.401191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c698db84684181b1e4247f72c5b2d5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:40.434568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2541c-b681-4c24-8161-a1a57bb0c705 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:40.434929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2541c-b681-4c24-8161-a1a57bb0c705 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:40.435287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2541c-b681-4c24-8161-a1a57bb0c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:40.483247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:40.504921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-23d2541c-b681-4c24-8161-a1a57bb0c705 None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:40.510605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-23d2541c-b681-4c24-8161-a1a57bb0c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:40.520457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c698db84684181b1e4247f72c5b2d5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:40.553377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c698db84684181b1e4247f72c5b2d5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:40.581631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c698db84684181b1e4247f72c5b2d5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:40.582843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4fa56b10-bf53-4bc7-bd19-f51da0af90ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c698db84684181b1e4247f72c5b2d5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:40.599915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 823/1649] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:51:39 2022] DELETE /identity/v3/users/52c698db84684181b1e4247f72c5b2d5 => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:51:40.614427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:40.614996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:40.615532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:40.681906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:40.684553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:40.782651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 827/1650] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:40.917681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953539fc-73f5-4341-918c-6b56b43c70aa None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:40.918206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953539fc-73f5-4341-918c-6b56b43c70aa None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:40.918624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953539fc-73f5-4341-918c-6b56b43c70aa None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:40.920457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 828/1651] 10.222.0.87 () {54 vars in 960 bytes} [Fri Jun 17 12:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:51:40.925540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ecffb-7cdb-4503-a2f7-808183feb021 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:40.926027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ecffb-7cdb-4503-a2f7-808183feb021 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:40.926442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ecffb-7cdb-4503-a2f7-808183feb021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:41.020153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-bd9ecffb-7cdb-4503-a2f7-808183feb021 None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:41.036903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 824/1652] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:41.040586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-bd9ecffb-7cdb-4503-a2f7-808183feb021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:41.175825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:41.177566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:41.328377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df42bf-74c6-441b-81ef-1fbbfa9b706d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:41.339906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:41.340735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 829/1653] 10.222.0.87 () {58 vars in 1057 bytes} [Fri Jun 17 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 17 12:51:41.342200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:41.342734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:41.343780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:41.348468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ecffb-7cdb-4503-a2f7-808183feb021 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:41.366838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:41.368857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:41.375744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ecffb-7cdb-4503-a2f7-808183feb021 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:41.392790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358133f-8330-4549-94d0-2210f6b4096a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:41.394381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358133f-8330-4549-94d0-2210f6b4096a service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:41.394659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358133f-8330-4549-94d0-2210f6b4096a service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:41.395410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358133f-8330-4549-94d0-2210f6b4096a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:41.399713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 825/1654] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:41.420204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358133f-8330-4549-94d0-2210f6b4096a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:41.426314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358133f-8330-4549-94d0-2210f6b4096a service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:41.456063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 830/1655] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:42.084579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:42.088437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:42.119235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02991af-03b1-45be-9501-7b995d5b1d3c service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:42.140380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:42.141629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:42.142072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:42.143564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] PATH_INFO: `/v3/users/7392dcf303da4771b6a40ab48dcffcd0` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:42.151439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7392dcf303da4771b6a40ab48dcffcd0)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:42.154769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:42.272608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7392dcf303da4771b6a40ab48dcffcd0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:42.296960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7392dcf303da4771b6a40ab48dcffcd0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:42.307533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:42.313163 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7392dcf303da4771b6a40ab48dcffcd0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:42.333806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7392dcf303da4771b6a40ab48dcffcd0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:42.334619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7392dcf303da4771b6a40ab48dcffcd0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:42.343089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 826/1656] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:51:42 2022] DELETE /identity/v3/users/7392dcf303da4771b6a40ab48dcffcd0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:42.348533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962a1a4-2a8b-4a55-a8fe-67a5e8dcf194 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:42.349012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962a1a4-2a8b-4a55-a8fe-67a5e8dcf194 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:42.349268 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962a1a4-2a8b-4a55-a8fe-67a5e8dcf194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:42.356674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:42.385157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-e962a1a4-2a8b-4a55-a8fe-67a5e8dcf194 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:42.387816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e962a1a4-2a8b-4a55-a8fe-67a5e8dcf194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:42.565342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 831/1657] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:42.631493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:42.632965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:42.761698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975b01-2cd8-4acf-8d1e-9ebc8329fbd9 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:42.784788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cca815-432b-44c5-bf29-1874f6202881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:42.785482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca815-432b-44c5-bf29-1874f6202881 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:42.785772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca815-432b-44c5-bf29-1874f6202881 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:42.786057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca815-432b-44c5-bf29-1874f6202881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:42.820313 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cca815-432b-44c5-bf29-1874f6202881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:42.821963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cca815-432b-44c5-bf29-1874f6202881 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:42.848785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 827/1658] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:43.187510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:43.187510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:43.187729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:43.231277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] MFA Rules not processed for user `a14c1230233e43789087e9239ffb91e2`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:43.234581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:43.375669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d80fab-2177-4107-99f7-9d015ab3f812 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:43.376543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d80fab-2177-4107-99f7-9d015ab3f812 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:43.376543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d80fab-2177-4107-99f7-9d015ab3f812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:43.441999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-31d80fab-2177-4107-99f7-9d015ab3f812 None None] MFA Rules not processed for user `efee42b434cd47d99984b85f677983c2`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:43.443680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-31d80fab-2177-4107-99f7-9d015ab3f812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:43.459514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 832/1659] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:43.475952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:43.478067 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:43.581866 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 828/1660] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:43.594403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:43.594924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:43.595336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:43.623293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] MFA Rules not processed for user `efee42b434cd47d99984b85f677983c2`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:43.624543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:43.650700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 829/1661] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:51:43.657694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:43.659086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:43.659624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cb737-5be9-47c6-a057-a47467e5e211 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:43.671649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:43.672280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:43.672571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:43.672851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:43.687083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449bd60-26b0-45db-9b76-cfe7fbdfbb87 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:43.698819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:43.700537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:43.716682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:43.717365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:43.719487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:43.719802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:43.727552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 833/1662] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:43.768756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:43.771226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:43.792322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:43.793776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:43.796641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 830/1663] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:43.817101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc45b0-b757-4708-9df5-147364732fb2 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:43.829974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:43.830876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:43.831329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:43.831773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] PATH_INFO: `/v3/projects/f796e354ac1d43318d0c3abc7a7201ae` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:43.833244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f796e354ac1d43318d0c3abc7a7201ae)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:43.834288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: WARNING py.warnings [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 12:51:43.834288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: warnings.warn(msg) Jun 17 12:51:43.834288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: Jun 17 12:51:43.835532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:43.879034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:43.970635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f796e354ac1d43318d0c3abc7a7201ae', 'request_id': 'req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.010092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f796e354ac1d43318d0c3abc7a7201ae', 'request_id': 'req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.034284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:44.036458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f796e354ac1d43318d0c3abc7a7201ae', 'request_id': 'req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.038050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:44.066925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] The token cache is being invalidate because project f796e354ac1d43318d0c3abc7a7201ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:51:44.067454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.069533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.079269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 834/1664] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:51:43 2022] DELETE /identity/v3/projects/f796e354ac1d43318d0c3abc7a7201ae => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:51:44.080660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:44.082892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec54210-927c-4b9f-bc5e-145644ef7d5f service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:44.088986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:44.298912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:44.299870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:44.302632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:44.302632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] PATH_INFO: `/v3/users/77eced3e20b3465d932e594b7149301c` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:44.309167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=77eced3e20b3465d932e594b7149301c)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:44.316382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:44.329504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:44.376240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff687ce-de6e-4bc2-bbf9-b92354f1cd81 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:44.425044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:44.425917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:44.426200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:44.426477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] PATH_INFO: `/v3/users/dc954e7b8a2c481d9da4b29b7a05f3e9` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:44.428141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc954e7b8a2c481d9da4b29b7a05f3e9)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:44.430318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:44.506204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77eced3e20b3465d932e594b7149301c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.541025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77eced3e20b3465d932e594b7149301c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.559975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77eced3e20b3465d932e594b7149301c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.565095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc954e7b8a2c481d9da4b29b7a05f3e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.578720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77eced3e20b3465d932e594b7149301c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.579981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af238cd9-5237-4624-bfcc-6e18e76307ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77eced3e20b3465d932e594b7149301c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.594675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 831/1665] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:51:44 2022] DELETE /identity/v3/users/77eced3e20b3465d932e594b7149301c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:44.598593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc954e7b8a2c481d9da4b29b7a05f3e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.611971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc954e7b8a2c481d9da4b29b7a05f3e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.622413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc954e7b8a2c481d9da4b29b7a05f3e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.623537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a5256d0e-4a50-455d-aaa1-ae095403e6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc954e7b8a2c481d9da4b29b7a05f3e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:44.631236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 835/1666] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:51:44 2022] DELETE /identity/v3/users/dc954e7b8a2c481d9da4b29b7a05f3e9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:45.345904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c8490-ca36-48a9-b0ce-87492d5f1296 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:45.346469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c8490-ca36-48a9-b0ce-87492d5f1296 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:45.346889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c8490-ca36-48a9-b0ce-87492d5f1296 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:45.348787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 832/1667] 10.222.0.87 () {54 vars in 912 bytes} [Fri Jun 17 12:51:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:51:45.355920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431990-794e-477c-a534-440b30dd2381 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:45.356451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431990-794e-477c-a534-440b30dd2381 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:45.356918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431990-794e-477c-a534-440b30dd2381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:45.398653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c8490-ca36-48a9-b0ce-87492d5f1296 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:45.401146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-285c8490-ca36-48a9-b0ce-87492d5f1296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:45.428296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9b431990-794e-477c-a534-440b30dd2381 None None] MFA Rules not processed for user `40f6ae0e5e164461b4c51b829be9a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:45.429544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9b431990-794e-477c-a534-440b30dd2381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:45.444985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:45.588580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c8490-ca36-48a9-b0ce-87492d5f1296 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:45.599770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 836/1668] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:45.603212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:45.604533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:45.605336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:45.605900 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] PATH_INFO: `/v3/projects/00745c24508d4170878017dcc2e0294b` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:45.607488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00745c24508d4170878017dcc2e0294b)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:45.609774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:45.630347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431990-794e-477c-a534-440b30dd2381 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:45.632590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9b431990-794e-477c-a534-440b30dd2381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:45.656701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431990-794e-477c-a534-440b30dd2381 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:45.671752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1966261-2215-4eea-a399-7a310ff4a9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:45.672771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1966261-2215-4eea-a399-7a310ff4a9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:45.673190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1966261-2215-4eea-a399-7a310ff4a9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:45.674487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1966261-2215-4eea-a399-7a310ff4a9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:45.698205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1966261-2215-4eea-a399-7a310ff4a9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:45.699972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1966261-2215-4eea-a399-7a310ff4a9e4 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:45.728070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 837/1669] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:45.743587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00745c24508d4170878017dcc2e0294b', 'request_id': 'req-103cdefe-94b0-464a-9582-20b78cf78242'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:45.767551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00745c24508d4170878017dcc2e0294b', 'request_id': 'req-103cdefe-94b0-464a-9582-20b78cf78242'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:45.781778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00745c24508d4170878017dcc2e0294b', 'request_id': 'req-103cdefe-94b0-464a-9582-20b78cf78242'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:45.822807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] The token cache is being invalidate because project 00745c24508d4170878017dcc2e0294b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:51:45.823138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:45.828990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:45.833150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 833/1670] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:51:45 2022] DELETE /identity/v3/projects/00745c24508d4170878017dcc2e0294b => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:45.846104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66710669-32a7-4b16-a8c5-a44bd79bea36 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:45.846402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66710669-32a7-4b16-a8c5-a44bd79bea36 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:45.846685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66710669-32a7-4b16-a8c5-a44bd79bea36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:45.881282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-66710669-32a7-4b16-a8c5-a44bd79bea36 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:45.884644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-66710669-32a7-4b16-a8c5-a44bd79bea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:45.919333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:45.921845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:45.945301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:45.988193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 838/1671] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:45.992021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051191f-06ee-48e1-b03c-3f8634c80fa8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:45.992021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051191f-06ee-48e1-b03c-3f8634c80fa8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:45.992436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051191f-06ee-48e1-b03c-3f8634c80fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.024269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4051191f-06ee-48e1-b03c-3f8634c80fa8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:46.025646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4051191f-06ee-48e1-b03c-3f8634c80fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:46.039557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 839/1672] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:46.045171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051191f-06ee-48e1-b03c-3f8634c80fa8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:46.055045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051191f-06ee-48e1-b03c-3f8634c80fa8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.070133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.072238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.072527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.072808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.073567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:46.073881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.076884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.085991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cdefe-94b0-464a-9582-20b78cf78242 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.100346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.101316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.101529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.101864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] PATH_INFO: `/v3/projects/802bb24d4c7c4040bd844a7b84698a71` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.107749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802bb24d4c7c4040bd844a7b84698a71)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.110054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 840/1673] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:46.111051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:46.111946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.134891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148946b9-b609-4625-b3b9-76de132b2bc7 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.152101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.152800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.153081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.153352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.154061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.156016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.219259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802bb24d4c7c4040bd844a7b84698a71', 'request_id': 'req-210e401a-af89-47a9-a2d2-5b105bfdead6'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.244141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802bb24d4c7c4040bd844a7b84698a71', 'request_id': 'req-210e401a-af89-47a9-a2d2-5b105bfdead6'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.255907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802bb24d4c7c4040bd844a7b84698a71', 'request_id': 'req-210e401a-af89-47a9-a2d2-5b105bfdead6'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.261773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 841/1674] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:46 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:46.268819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:46.301021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] The token cache is being invalidate because project 802bb24d4c7c4040bd844a7b84698a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:51:46.302499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.307026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.317630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 834/1675] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:51:45 2022] DELETE /identity/v3/projects/802bb24d4c7c4040bd844a7b84698a71 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:46.319041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:46.326989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:46.479193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc41cd0-c4f7-4ecc-997d-02222ffb578e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.516081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.516822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.517109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.529182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.530112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.540098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.567770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:46.598767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 842/1676] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:46 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:46.607981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:46.610971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e401a-af89-47a9-a2d2-5b105bfdead6 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.612540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:46.630771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.632075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.632476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.632886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] PATH_INFO: `/v3/users/ba13730fb72b4db1bbd531cd5c565e3a` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.634926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba13730fb72b4db1bbd531cd5c565e3a)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.638361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.647570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7029e-12ac-43a6-924b-c8dc1736445b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.658984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.659823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.660189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.660482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.661270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.667595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.699062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 843/1677] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:46.701991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:46.731345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f065bdc-ccb4-4799-938b-e46159d0ad7f admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.739193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.740205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.740668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.741322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.742334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.744445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.772026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:46.776401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba13730fb72b4db1bbd531cd5c565e3a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.783635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 844/1678] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:46.786251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:46.811685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721be06c-8a8d-49f7-bbd8-7dd5e0b72ee6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:46.819516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4f499-b941-43c6-b695-dbf2f5c81191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:46.820601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4f499-b941-43c6-b695-dbf2f5c81191 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.820947 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4f499-b941-43c6-b695-dbf2f5c81191 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.821294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4f499-b941-43c6-b695-dbf2f5c81191 admin admin] PATH_INFO: `/v3/projects/01072eb5a4aa42b0842d3f4faecabe8f/users/c1686304f4c14f17abc01b06cce6f864/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.833108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba13730fb72b4db1bbd531cd5c565e3a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.846516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4f499-b941-43c6-b695-dbf2f5c81191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01072eb5a4aa42b0842d3f4faecabe8f, user_id=c1686304f4c14f17abc01b06cce6f864, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:46.849566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4f499-b941-43c6-b695-dbf2f5c81191 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:46.856368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba13730fb72b4db1bbd531cd5c565e3a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.883499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba13730fb72b4db1bbd531cd5c565e3a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.884633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba13730fb72b4db1bbd531cd5c565e3a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:46.894188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 845/1679] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:46 2022] PUT /identity/v3/projects/01072eb5a4aa42b0842d3f4faecabe8f/users/c1686304f4c14f17abc01b06cce6f864/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:46.910448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678e4c4-0a04-40db-b375-61ce1650fa46 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:46.910906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678e4c4-0a04-40db-b375-61ce1650fa46 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:46.911298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678e4c4-0a04-40db-b375-61ce1650fa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:46.915627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 835/1680] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:51:46 2022] DELETE /identity/v3/users/ba13730fb72b4db1bbd531cd5c565e3a => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:46.985740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a678e4c4-0a04-40db-b375-61ce1650fa46 None None] MFA Rules not processed for user `c1686304f4c14f17abc01b06cce6f864`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:46.990138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a678e4c4-0a04-40db-b375-61ce1650fa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:47.118480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 846/1681] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:47.120587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:47.122279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:47.282180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513601f-a2bd-4e7e-a5ea-6cd554ec944e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:47.295578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:47.296523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:47.296813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:47.297174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:47.297892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:47.299940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:47.371355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 836/1682] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:47 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:47.374451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678e4c4-0a04-40db-b375-61ce1650fa46 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:47.405544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678e4c4-0a04-40db-b375-61ce1650fa46 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:47.457649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:47.459552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:47.560926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:47.561662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:47.561950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:47.562229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:47.562963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:47.568104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:47.629810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 847/1683] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:47 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:47.631527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:47.635659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:47.667283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f18a-f39c-44df-9ef5-4599fe826298 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:47.678366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:47.679569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:47.680120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:47.680735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] PATH_INFO: `/v3/projects/78332df60e584f23ac09528633d288d0` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:47.684222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=78332df60e584f23ac09528633d288d0)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:47.686355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:47.690677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95403d-3652-430a-a68d-391390132514 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:47.715547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:47.716333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:47.716649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:47.717047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:47.725290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:47.748322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:47.750051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:47.768348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:47.806255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78332df60e584f23ac09528633d288d0', 'request_id': 'req-885a8ec8-50a2-4c94-9aec-b05599840b6d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:47.825861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78332df60e584f23ac09528633d288d0', 'request_id': 'req-885a8ec8-50a2-4c94-9aec-b05599840b6d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:47.833685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 837/1684] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:47.834895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78332df60e584f23ac09528633d288d0', 'request_id': 'req-885a8ec8-50a2-4c94-9aec-b05599840b6d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:47.835817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:47.864665 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] The token cache is being invalidate because project 78332df60e584f23ac09528633d288d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:51:47.864962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:47.866496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-885a8ec8-50a2-4c94-9aec-b05599840b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:47.868768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 848/1685] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:51:47 2022] DELETE /identity/v3/projects/78332df60e584f23ac09528633d288d0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:47.875043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c5902-6ec8-4e7e-b8a6-408773e509df None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:47.875274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c5902-6ec8-4e7e-b8a6-408773e509df None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:47.875633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c5902-6ec8-4e7e-b8a6-408773e509df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:47.878092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cd8e-05b6-4e82-a8d4-7410103c78f2 service placement] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:47.928948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f37c5902-6ec8-4e7e-b8a6-408773e509df None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:47.930381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f37c5902-6ec8-4e7e-b8a6-408773e509df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:47.988367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:48.058145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:48.059049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.059555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.060203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.061304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:48.063745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:48.085802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 838/1686] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:48.101321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:48.105830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:48.108736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 849/1687] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:48.123178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1be751-aaf1-4e06-8857-ead3981c2b0f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.123497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1be751-aaf1-4e06-8857-ead3981c2b0f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.123777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1be751-aaf1-4e06-8857-ead3981c2b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.144578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6486-4af0-471e-89da-a10c219ca91e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:48.153685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4b1be751-aaf1-4e06-8857-ead3981c2b0f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:48.154385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:48.155442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.155818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.156175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] PATH_INFO: `/v3/projects/b42ae2e961344774b3e1ef9c74fd1e16/users/fd9e38583e8647ad9b1ae8dd7791e1bf/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.156433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4b1be751-aaf1-4e06-8857-ead3981c2b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:48.184575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 850/1688] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:48.185591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1be751-aaf1-4e06-8857-ead3981c2b0f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:48.195720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42ae2e961344774b3e1ef9c74fd1e16, user_id=fd9e38583e8647ad9b1ae8dd7791e1bf, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:48.208552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1be751-aaf1-4e06-8857-ead3981c2b0f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:48.211187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:48.217597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:48.218297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.218607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.218850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.219633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:48.219971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:48.221604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:48.241153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 839/1689] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:48 2022] PUT /identity/v3/projects/b42ae2e961344774b3e1ef9c74fd1e16/users/fd9e38583e8647ad9b1ae8dd7791e1bf/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:48.247070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:48.253782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 851/1690] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:48.258344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:48.264644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:48.292199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8b23f-a627-406f-8259-2276971884c2 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:48.428003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9310328-c92b-441b-b5a0-a6cc3aea3474 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:48.475002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:48.475734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.476018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.476292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.477370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:48.487784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:48.523466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:48.524615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 840/1691] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:48.525654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:48.526214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.526512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.526798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] PATH_INFO: `/v3/projects/26dc5562058a400c8965f2739493d3e8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.533437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=26dc5562058a400c8965f2739493d3e8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:48.538262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:48.546963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577a3b4-77eb-4ed8-aa98-71afb40580bf admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:48.562238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acc4c6-8f3b-4883-b88d-5699550acea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:48.562238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acc4c6-8f3b-4883-b88d-5699550acea8 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.562544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acc4c6-8f3b-4883-b88d-5699550acea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.562999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acc4c6-8f3b-4883-b88d-5699550acea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.563952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acc4c6-8f3b-4883-b88d-5699550acea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:48.568524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acc4c6-8f3b-4883-b88d-5699550acea8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:48.642141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 841/1692] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:48 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:48.646993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:48.647283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:48.649147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:48.658461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:48.689098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26dc5562058a400c8965f2739493d3e8', 'request_id': 'req-38e53e75-c00d-4783-8258-70fe8c63efbb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:48.730138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26dc5562058a400c8965f2739493d3e8', 'request_id': 'req-38e53e75-c00d-4783-8258-70fe8c63efbb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:48.739241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26dc5562058a400c8965f2739493d3e8', 'request_id': 'req-38e53e75-c00d-4783-8258-70fe8c63efbb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:48.762899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] The token cache is being invalidate because project 26dc5562058a400c8965f2739493d3e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:51:48.763563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] MFA Rules not processed for user `efee42b434cd47d99984b85f677983c2`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:48.764448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:48.764550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:48.765673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:51:48.773340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 852/1693] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:51:48 2022] DELETE /identity/v3/projects/26dc5562058a400c8965f2739493d3e8 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:48.774924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:48.780823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:48.939753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 842/1694] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:48.939753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:48.941180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:48.984190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e53e75-c00d-4783-8258-70fe8c63efbb admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:48.999681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd944f4-ffaf-455e-a99f-7a5928b57d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:49.001086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd944f4-ffaf-455e-a99f-7a5928b57d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.001498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd944f4-ffaf-455e-a99f-7a5928b57d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.001977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd944f4-ffaf-455e-a99f-7a5928b57d7d admin admin] PATH_INFO: `/v3/projects/b42ae2e961344774b3e1ef9c74fd1e16/users/fd9e38583e8647ad9b1ae8dd7791e1bf/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.004599 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd944f4-ffaf-455e-a99f-7a5928b57d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42ae2e961344774b3e1ef9c74fd1e16, user_id=fd9e38583e8647ad9b1ae8dd7791e1bf, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:49.015074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:49.016956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd944f4-ffaf-455e-a99f-7a5928b57d7d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:49.047362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 853/1695] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:48 2022] PUT /identity/v3/projects/b42ae2e961344774b3e1ef9c74fd1e16/users/fd9e38583e8647ad9b1ae8dd7791e1bf/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:49.059923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ecf72-535b-403f-b4ae-da4ccd22c349 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.062607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ecf72-535b-403f-b4ae-da4ccd22c349 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.063057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ecf72-535b-403f-b4ae-da4ccd22c349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.117419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cb220-e2c8-4abd-bcf1-fc171c7dc367 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:49.119003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b97ecf72-535b-403f-b4ae-da4ccd22c349 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:49.125032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b97ecf72-535b-403f-b4ae-da4ccd22c349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.235178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698e843-6f39-4436-b294-584ebe3c3ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:49.235997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698e843-6f39-4436-b294-584ebe3c3ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.236285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698e843-6f39-4436-b294-584ebe3c3ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.236564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698e843-6f39-4436-b294-584ebe3c3ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.237291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698e843-6f39-4436-b294-584ebe3c3ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:49.242679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698e843-6f39-4436-b294-584ebe3c3ad2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:49.293922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 843/1696] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:48 2022] POST /identity/v3/users => generated 283 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:49.298367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a515fd3-905f-4617-a528-bcc2af5d4f94 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.298367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a515fd3-905f-4617-a528-bcc2af5d4f94 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.298591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a515fd3-905f-4617-a528-bcc2af5d4f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.322830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 854/1697] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:49.329322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b04986-c0d3-42b2-a5bd-5cb08098f1d6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.329322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b04986-c0d3-42b2-a5bd-5cb08098f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.329322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b04986-c0d3-42b2-a5bd-5cb08098f1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.361414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-1a515fd3-905f-4617-a528-bcc2af5d4f94 None None] MFA Rules not processed for user `fd9e38583e8647ad9b1ae8dd7791e1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:49.362821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1a515fd3-905f-4617-a528-bcc2af5d4f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.432560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-71b04986-c0d3-42b2-a5bd-5cb08098f1d6 None None] MFA Rules not processed for user `f3bfb04b34ed4761bb09be7b29d1ec8f`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:49.433708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-71b04986-c0d3-42b2-a5bd-5cb08098f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.484464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 844/1698] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:49.486859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.487290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.487842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.511126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:49.515617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] MFA Rules not processed for user `9863e406790b413faf022554ba4b8dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:49.520422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.540053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 855/1699] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:49.551509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b04986-c0d3-42b2-a5bd-5cb08098f1d6 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:49.567220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b04986-c0d3-42b2-a5bd-5cb08098f1d6 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:49.575075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9344fa-5793-4dc3-afea-74dea6361623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:49.576041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9344fa-5793-4dc3-afea-74dea6361623 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.576504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9344fa-5793-4dc3-afea-74dea6361623 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.576918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9344fa-5793-4dc3-afea-74dea6361623 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.577849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9344fa-5793-4dc3-afea-74dea6361623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:49.579852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9344fa-5793-4dc3-afea-74dea6361623 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:49.597056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 856/1700] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:49.597056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146510-d070-4bec-a4bc-c5e9f1ad797f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.597056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146510-d070-4bec-a4bc-c5e9f1ad797f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.597933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146510-d070-4bec-a4bc-c5e9f1ad797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.622557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a1146510-d070-4bec-a4bc-c5e9f1ad797f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:49.624403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a1146510-d070-4bec-a4bc-c5e9f1ad797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.631221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:49.638120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 857/1701] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:49.640057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.640433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.641419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.659937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 845/1702] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:49.666773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:49.675064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.678607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] MFA Rules not processed for user `a17b8a752d78431dbde11be6d0812de3`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:49.680216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.796885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 858/1703] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:49.797401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:49.811246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:49.816065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a359f1-1729-4268-ae3b-9cb5e98a95b0 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:49.825788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:49.826479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:49.826906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:49.827188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:49.834608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:50.009367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:50.014645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:50.035857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22feba4-5d36-42b6-869e-a50f62a959f7 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:50.037938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 846/1704] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:50.040110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:50.049553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:50.050349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.050760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.051214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.069697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:50.071763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:50.072804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84229e-ee15-44ff-b08d-eb423bc249bf service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:50.078181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7095f-a982-4fff-8e74-2a41dc6976ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:50.078891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7095f-a982-4fff-8e74-2a41dc6976ba admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.079179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7095f-a982-4fff-8e74-2a41dc6976ba admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.079585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7095f-a982-4fff-8e74-2a41dc6976ba admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.080343 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7095f-a982-4fff-8e74-2a41dc6976ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:50.081995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7095f-a982-4fff-8e74-2a41dc6976ba admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:50.096377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 859/1705] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:50.097409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:50.102661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 847/1706] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:50.106524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a240221-1b5f-4afd-ae03-49c0eb1d37bd None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.118300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a240221-1b5f-4afd-ae03-49c0eb1d37bd None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.120312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a240221-1b5f-4afd-ae03-49c0eb1d37bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.127331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c7fe1-aa77-4b50-b937-bc428ece2d44 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:50.140262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519f6b3-4c51-441d-8078-6974b4924dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:50.141598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519f6b3-4c51-441d-8078-6974b4924dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.141994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519f6b3-4c51-441d-8078-6974b4924dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.142381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519f6b3-4c51-441d-8078-6974b4924dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.143253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519f6b3-4c51-441d-8078-6974b4924dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:51:50.143688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519f6b3-4c51-441d-8078-6974b4924dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:50.149674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519f6b3-4c51-441d-8078-6974b4924dc1 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:50.170619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-6a240221-1b5f-4afd-ae03-49c0eb1d37bd None None] MFA Rules not processed for user `a17b8a752d78431dbde11be6d0812de3`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:50.176214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6a240221-1b5f-4afd-ae03-49c0eb1d37bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:50.200109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 848/1707] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:50.205950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 860/1708] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:51:50.207675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a240221-1b5f-4afd-ae03-49c0eb1d37bd None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:50.208687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1b732-d006-4d2c-8f6d-2f4c61ebad4d None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.210393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1b732-d006-4d2c-8f6d-2f4c61ebad4d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.210951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1b732-d006-4d2c-8f6d-2f4c61ebad4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.247667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d7c1b732-d006-4d2c-8f6d-2f4c61ebad4d None None] MFA Rules not processed for user `a14c1230233e43789087e9239ffb91e2`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:50.249297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d7c1b732-d006-4d2c-8f6d-2f4c61ebad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:50.252214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a240221-1b5f-4afd-ae03-49c0eb1d37bd None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:50.261821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:50.262754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.263040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.263320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] PATH_INFO: `/v3/projects/0b6e2195f5c64d52a7bccd68e4e751d5/users/4c5b744438bb429c9111f3de0c5aa953/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.289352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:50.304965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6e2195f5c64d52a7bccd68e4e751d5, user_id=4c5b744438bb429c9111f3de0c5aa953, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:50.309436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:50.337985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 849/1709] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:50 2022] PUT /identity/v3/projects/0b6e2195f5c64d52a7bccd68e4e751d5/users/4c5b744438bb429c9111f3de0c5aa953/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:50.339323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:50.384640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 861/1710] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:50.387910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05210d-0d93-473c-b3ee-82e585c96ddb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.388641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05210d-0d93-473c-b3ee-82e585c96ddb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.389726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05210d-0d93-473c-b3ee-82e585c96ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.473848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8c05210d-0d93-473c-b3ee-82e585c96ddb None None] MFA Rules not processed for user `4c5b744438bb429c9111f3de0c5aa953`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:50.476715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8c05210d-0d93-473c-b3ee-82e585c96ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:50.540252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8a3b7-f862-4692-a565-538547280be7 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:50.557355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:50.558159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.558962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.559617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.560494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:50.567305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:50.596439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:50.635194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 850/1711] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:50 2022] POST /identity/v3/projects => generated 379 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:50.636403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:50.638333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:50.681885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 862/1712] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:50.685153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05210d-0d93-473c-b3ee-82e585c96ddb None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:50.740619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05210d-0d93-473c-b3ee-82e585c96ddb None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:50.754099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24da30-aed8-49d9-a2b4-08e335390fc0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:50.792096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:50.793431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:50.794025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:50.794391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:50.964078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:50.966959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:50.987219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 851/1713] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:50.989157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:51.010035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:51.011887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.015736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.027270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.033951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:51.046389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:51.119881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 863/1714] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:50 2022] POST /identity/v3/users => generated 290 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:51.121093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:51.145974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fefc90-5ea3-45e1-9e38-96f6bf55704c service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:51.151915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:51.153034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.153326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.153777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.155479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:51.159361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:51.160504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac476d87-3725-437b-8f6a-3c2f6d5cabcc admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:51.184406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:51.185403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.185521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.186723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.187233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:51.188950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:51.216685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 852/1715] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:50 2022] POST /identity/v3/projects => generated 363 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:51.222191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:51.233498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 864/1716] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:51.238915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:51.258190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:51.277189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915306b-375e-41c1-8b7e-60d4df65cb0b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:51.391515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafc4df-a06d-4285-87fc-00f6d02a7721 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:51.413823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:51.429603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:51.434688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.434997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.435275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.439011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:51.449429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:51.466394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:51.467056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.468910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.468910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.469600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:51.471275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:51.502220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 865/1717] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:51.511133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:51.537252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 853/1718] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:51 2022] POST /identity/v3/users => generated 288 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:51.553606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:51.567526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a3611-2852-4f4e-ac8c-4dd7f1aee7bb admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:51.583970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097e7c-a358-47a7-8c31-267b64ab678d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:51.590870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:51.591906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.596520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.596520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] PATH_INFO: `/v3/projects/2203638a5cdb44e7b427ae4c15244d2e/users/b39f809d6ee343e9b8c90083d5f37dc6/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.604797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a920c9d-b972-41e0-8b7c-57952cdc1206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:51.607290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920c9d-b972-41e0-8b7c-57952cdc1206 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.608746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920c9d-b972-41e0-8b7c-57952cdc1206 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.609208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920c9d-b972-41e0-8b7c-57952cdc1206 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.612147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a920c9d-b972-41e0-8b7c-57952cdc1206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:51.616897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a920c9d-b972-41e0-8b7c-57952cdc1206 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:51.647746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 854/1719] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:51.654306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5fbb0-0afa-48c9-9fa9-2b93304b7b55 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.654879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5fbb0-0afa-48c9-9fa9-2b93304b7b55 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.656371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5fbb0-0afa-48c9-9fa9-2b93304b7b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.684311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2203638a5cdb44e7b427ae4c15244d2e, user_id=b39f809d6ee343e9b8c90083d5f37dc6, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:51.697815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:51.713114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-cac5fbb0-0afa-48c9-9fa9-2b93304b7b55 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:51.717768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cac5fbb0-0afa-48c9-9fa9-2b93304b7b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:51.761261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 866/1720] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:51 2022] PUT /identity/v3/projects/2203638a5cdb44e7b427ae4c15244d2e/users/b39f809d6ee343e9b8c90083d5f37dc6/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:51.763881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:51.926038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 855/1721] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:51:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:51.928058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b2584-99fc-40b8-9e27-aebcb2d39181 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:51.928244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b2584-99fc-40b8-9e27-aebcb2d39181 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:51.928767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b2584-99fc-40b8-9e27-aebcb2d39181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:51.977768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-1d3b2584-99fc-40b8-9e27-aebcb2d39181 None None] MFA Rules not processed for user `b39f809d6ee343e9b8c90083d5f37dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:51.979309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1d3b2584-99fc-40b8-9e27-aebcb2d39181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:52.031215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af88e-7422-4e1b-88c2-b645713c5b90 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:52.056299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:52.057204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:52.057492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:52.057772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] PATH_INFO: `/v3/projects/5715dffef7d646c3a72a896a643c6e56/users/901dda0ca7864044b68d1e5ddefbc0fd/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:52.143916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5715dffef7d646c3a72a896a643c6e56, user_id=901dda0ca7864044b68d1e5ddefbc0fd, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:52.147534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:52.159115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 856/1722] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:52.167357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1e6f0-8582-48c4-8d26-45bc2f7e43f4 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:52.167669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1e6f0-8582-48c4-8d26-45bc2f7e43f4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:52.167952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1e6f0-8582-48c4-8d26-45bc2f7e43f4 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:52.175634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 867/1723] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:51 2022] PUT /identity/v3/projects/5715dffef7d646c3a72a896a643c6e56/users/901dda0ca7864044b68d1e5ddefbc0fd/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:52.181471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:52.182248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 857/1724] 10.222.0.87 () {54 vars in 912 bytes} [Fri Jun 17 12:51:52 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:51:52.187257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1e6f0-8582-48c4-8d26-45bc2f7e43f4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:52.288736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:52.414720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1e6f0-8582-48c4-8d26-45bc2f7e43f4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:52.425675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a537a-ff49-46c0-af7f-3f0dd3ca25c1 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:52.429198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b5a5e-0623-4b12-9dbf-747174b35cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:52.431580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b5a5e-0623-4b12-9dbf-747174b35cdb admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:52.431935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b5a5e-0623-4b12-9dbf-747174b35cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:52.432219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b5a5e-0623-4b12-9dbf-747174b35cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:52.433057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b5a5e-0623-4b12-9dbf-747174b35cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:52.434873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b5a5e-0623-4b12-9dbf-747174b35cdb admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:52.452912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 858/1725] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:52.453083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:52.453889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:52.454179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:52.454304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed7bec-e0bb-47ea-96a2-4499634076f1 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:52.454455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:52.454752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed7bec-e0bb-47ea-96a2-4499634076f1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:52.455039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed7bec-e0bb-47ea-96a2-4499634076f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:52.455166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:52.457728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:52.516125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-66ed7bec-e0bb-47ea-96a2-4499634076f1 None None] MFA Rules not processed for user `40f6ae0e5e164461b4c51b829be9a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:52.519879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-66ed7bec-e0bb-47ea-96a2-4499634076f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:52.583437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 868/1726] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:51:52 2022] POST /identity/v3/projects => generated 379 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:52.584588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:52.586792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:52.663272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:52.722764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 859/1727] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:52.725529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed7bec-e0bb-47ea-96a2-4499634076f1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:52.760167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed7bec-e0bb-47ea-96a2-4499634076f1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:52.817161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a7c7-a8dc-4ec3-9596-347352e20195 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:52.855781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:52.864478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:52.865002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:52.867050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:52.901090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:52.902551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:52.902883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:52.903160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] PATH_INFO: `/v3/projects/5715dffef7d646c3a72a896a643c6e56/users/901dda0ca7864044b68d1e5ddefbc0fd/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:52.905781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5715dffef7d646c3a72a896a643c6e56, user_id=901dda0ca7864044b68d1e5ddefbc0fd, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:52.909421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:52.942541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 860/1728] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:52 2022] PUT /identity/v3/projects/5715dffef7d646c3a72a896a643c6e56/users/901dda0ca7864044b68d1e5ddefbc0fd/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:52.943652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:53.048272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.054976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.082311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 869/1729] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 12:51:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 12:51:53.083288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:53.087095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:53.101500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3fee1-e790-4447-9975-c2682439cbcb admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:53.124959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f107d8-269d-4270-b64e-5fe7f6683503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:53.126894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f107d8-269d-4270-b64e-5fe7f6683503 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.127325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f107d8-269d-4270-b64e-5fe7f6683503 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.127757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f107d8-269d-4270-b64e-5fe7f6683503 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.131113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f107d8-269d-4270-b64e-5fe7f6683503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.133333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f107d8-269d-4270-b64e-5fe7f6683503 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.166105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:53.194352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e8011-3670-499d-a46f-1ae1c84e4fc1 service swift] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:53.196260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 861/1730] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:51:52 2022] POST /identity/v3/users => generated 296 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:51:53.198386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.198900 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.199289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.273301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] MFA Rules not processed for user `901dda0ca7864044b68d1e5ddefbc0fd`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:53.274822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:53.323530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:53.324285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.324612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.325104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.336830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.340320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.371395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 870/1731] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:53.373051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:53.405014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54da3b-b300-462b-a499-1150175b54a4 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:53.416422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:53.418510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.418801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.419086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.421157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.425053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.440251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 862/1732] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:53.449220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:53.450951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:53.458623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 871/1733] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:53.458623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:53.489729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7810d-d13b-499e-a040-100a3437ad5e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:53.499789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:53.501883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.505100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.505560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] PATH_INFO: `/v3/projects/e84b22d3fb0f49e98399223fa3c69195/users/2c9a9e752cba460aabaeac31519db01e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.532969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:53.551002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84b22d3fb0f49e98399223fa3c69195, user_id=2c9a9e752cba460aabaeac31519db01e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.554553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.590134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 872/1734] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:53 2022] PUT /identity/v3/projects/e84b22d3fb0f49e98399223fa3c69195/users/2c9a9e752cba460aabaeac31519db01e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:53.591517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:53.665011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ae618-0578-4f7a-a4c8-7fbe2a2b76c8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:53.694989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f29a5-d8aa-4af0-8af6-4bfcd5a12601 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:53.703640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:53.704529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.704920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.705300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.706860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.709320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.766632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 873/1735] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:51:53.767858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:53.779144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:53.780020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.780797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.780797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.801116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be85d76-a39f-4e00-8c3f-04e9c3d149c7 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:53.818666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa02e84-017e-4ace-9f58-a3f2593a0b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:53.820845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa02e84-017e-4ace-9f58-a3f2593a0b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.821651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa02e84-017e-4ace-9f58-a3f2593a0b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.822227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa02e84-017e-4ace-9f58-a3f2593a0b90 admin admin] PATH_INFO: `/v3/projects/e84b22d3fb0f49e98399223fa3c69195/users/2c9a9e752cba460aabaeac31519db01e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.827363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa02e84-017e-4ace-9f58-a3f2593a0b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84b22d3fb0f49e98399223fa3c69195, user_id=2c9a9e752cba460aabaeac31519db01e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.832234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa02e84-017e-4ace-9f58-a3f2593a0b90 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.874342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 874/1736] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:51:53 2022] PUT /identity/v3/projects/e84b22d3fb0f49e98399223fa3c69195/users/2c9a9e752cba460aabaeac31519db01e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:51:53.878925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2313f-b337-4889-8882-61d7f06131b0 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:53.879541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2313f-b337-4889-8882-61d7f06131b0 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:53.882542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2313f-b337-4889-8882-61d7f06131b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:53.896125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:53.935167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:53.936786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:53.974665 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0ab2313f-b337-4889-8882-61d7f06131b0 None None] MFA Rules not processed for user `2c9a9e752cba460aabaeac31519db01e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:53.976000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0ab2313f-b337-4889-8882-61d7f06131b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:54.021599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 863/1737] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:54.133594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 875/1738] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:54.145971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:54.157131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e9fd6-1b9b-4d32-baa2-50905d9fe80f service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:54.222816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2313f-b337-4889-8882-61d7f06131b0 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:54.327140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:54.328468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:54.330594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:54.331503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:54.361317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2313f-b337-4889-8882-61d7f06131b0 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:54.367753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bba97c-0119-4c29-974f-74308d010301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:54.368604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bba97c-0119-4c29-974f-74308d010301 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:54.368821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:54.370512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:54.390083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 864/1739] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:54.391476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bba97c-0119-4c29-974f-74308d010301 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:54.391906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bba97c-0119-4c29-974f-74308d010301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:54.474728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:51:54.546184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:54.557496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05908-27ee-40b1-8437-4dfde8421f44 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:54.569565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:54.570249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:54.570537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:54.570813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:54.592331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bba97c-0119-4c29-974f-74308d010301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:54.596135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bba97c-0119-4c29-974f-74308d010301 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:54.609574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 876/1740] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:54.776406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:54.777985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:54.806061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 865/1741] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:55.800895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cacbd5-0f43-40c3-929c-abe51e6e2889 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:55.802994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cacbd5-0f43-40c3-929c-abe51e6e2889 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:55.803624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cacbd5-0f43-40c3-929c-abe51e6e2889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:55.862715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-90cacbd5-0f43-40c3-929c-abe51e6e2889 None None] MFA Rules not processed for user `40f6ae0e5e164461b4c51b829be9a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:55.874503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-90cacbd5-0f43-40c3-929c-abe51e6e2889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:56.072919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 877/1742] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 17 12:51:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:51:56.091590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:56.093149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:56.182978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06fbe0-bf04-4bd6-aefb-eaf3a29fe5a1 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:56.224520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f03a65-3f41-4641-a548-d5cecbcd031a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:56.242707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f03a65-3f41-4641-a548-d5cecbcd031a service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:56.245595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f03a65-3f41-4641-a548-d5cecbcd031a service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:56.246904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f03a65-3f41-4641-a548-d5cecbcd031a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:56.273423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cacbd5-0f43-40c3-929c-abe51e6e2889 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:51:56.297378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cacbd5-0f43-40c3-929c-abe51e6e2889 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:51:56.309961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f03a65-3f41-4641-a548-d5cecbcd031a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:56.319164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f03a65-3f41-4641-a548-d5cecbcd031a service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:56.321613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbaddb-a266-4e39-b58d-ed452961e87b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:51:56.329633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbaddb-a266-4e39-b58d-ed452961e87b service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:56.333691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbaddb-a266-4e39-b58d-ed452961e87b service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:56.352360 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbaddb-a266-4e39-b58d-ed452961e87b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:56.412450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 866/1743] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:51:56.709994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbaddb-a266-4e39-b58d-ed452961e87b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:51:56.716127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbaddb-a266-4e39-b58d-ed452961e87b service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:51:56.784273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 878/1744] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:51:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:51:56.942960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b3f1-baeb-42fb-afcb-3905756b3bbb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:56.943841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b3f1-baeb-42fb-afcb-3905756b3bbb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:56.944451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b3f1-baeb-42fb-afcb-3905756b3bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:56.982788 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-eed1b3f1-baeb-42fb-afcb-3905756b3bbb None None] MFA Rules not processed for user `f3bfb04b34ed4761bb09be7b29d1ec8f`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:56.984616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-eed1b3f1-baeb-42fb-afcb-3905756b3bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:51:57.285074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 867/1745] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:51:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:51:59.939746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63838c-2a69-4574-ae49-ba664fa4065e None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:59.941313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63838c-2a69-4574-ae49-ba664fa4065e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:59.941794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63838c-2a69-4574-ae49-ba664fa4065e None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:59.944457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 879/1746] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:51:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:51:59.953513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ecf31-2c55-4124-a58f-e528499e1876 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:51:59.954095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ecf31-2c55-4124-a58f-e528499e1876 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:51:59.954290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ecf31-2c55-4124-a58f-e528499e1876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:51:59.979006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c16ecf31-2c55-4124-a58f-e528499e1876 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:51:59.980653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c16ecf31-2c55-4124-a58f-e528499e1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:00.011157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:00.143206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 868/1747] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:51:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:00.154204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63838c-2a69-4574-ae49-ba664fa4065e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:00.172189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63838c-2a69-4574-ae49-ba664fa4065e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:00.181188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b9243-19e6-44ea-a4cb-6adab7310ef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:00.181880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b9243-19e6-44ea-a4cb-6adab7310ef2 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:00.182326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b9243-19e6-44ea-a4cb-6adab7310ef2 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:00.182609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b9243-19e6-44ea-a4cb-6adab7310ef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:00.195264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b9243-19e6-44ea-a4cb-6adab7310ef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:00.197183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b9243-19e6-44ea-a4cb-6adab7310ef2 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:00.222310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 880/1748] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:02.165699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ecf31-2c55-4124-a58f-e528499e1876 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:02.235426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ecf31-2c55-4124-a58f-e528499e1876 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:02.240392 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:02.241318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:02.241599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:02.241871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] PATH_INFO: `/v3/users/4c5b744438bb429c9111f3de0c5aa953` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:02.243082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c5b744438bb429c9111f3de0c5aa953)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:02.245861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:02.300460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c5b744438bb429c9111f3de0c5aa953'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:02.313040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c5b744438bb429c9111f3de0c5aa953'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:02.319794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c5b744438bb429c9111f3de0c5aa953'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:02.330950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c5b744438bb429c9111f3de0c5aa953'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:02.331975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c5b744438bb429c9111f3de0c5aa953'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:02.340128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 869/1749] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:02 2022] DELETE /identity/v3/users/4c5b744438bb429c9111f3de0c5aa953 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:02.345230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:02.345759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:02.345848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:02.364258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:02.365484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:02.418981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:02.458483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 881/1750] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:02.468740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:02.470211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:02.505751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:02.541536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f4d47-d0ca-4ce3-89fb-bbc17b1ad26c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:02.555936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:02.556861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:02.557234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:02.557516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:02.575750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:02.577360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:02.592992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 870/1751] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:02.948768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:02.950407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:02.987475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1182e-adce-4698-b74d-5d1a9eb0d74a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:03.007728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:03.008561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:03.008856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:03.009136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] PATH_INFO: `/v3/projects/0b6e2195f5c64d52a7bccd68e4e751d5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:03.010551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6e2195f5c64d52a7bccd68e4e751d5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:03.012585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:03.092017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6e2195f5c64d52a7bccd68e4e751d5', 'request_id': 'req-15b75724-2a02-4383-b143-6ca3bfd1cb59'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.114253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6e2195f5c64d52a7bccd68e4e751d5', 'request_id': 'req-15b75724-2a02-4383-b143-6ca3bfd1cb59'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.127342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6e2195f5c64d52a7bccd68e4e751d5', 'request_id': 'req-15b75724-2a02-4383-b143-6ca3bfd1cb59'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.157078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] The token cache is being invalidate because project 0b6e2195f5c64d52a7bccd68e4e751d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:03.157447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.158257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.160348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 882/1752] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:02 2022] DELETE /identity/v3/projects/0b6e2195f5c64d52a7bccd68e4e751d5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:03.164546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:03.187514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382db27-a674-4d46-8e9a-708d4f52cf05 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:03.277678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:03.278532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:03.278817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:03.279097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] PATH_INFO: `/v3/users/901dda0ca7864044b68d1e5ddefbc0fd` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:03.280393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=901dda0ca7864044b68d1e5ddefbc0fd)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:03.282385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:03.345700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901dda0ca7864044b68d1e5ddefbc0fd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.360649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901dda0ca7864044b68d1e5ddefbc0fd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.367658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901dda0ca7864044b68d1e5ddefbc0fd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.376434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901dda0ca7864044b68d1e5ddefbc0fd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.377080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-10c017ef-60f6-4abf-a447-b81feea810d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901dda0ca7864044b68d1e5ddefbc0fd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.383655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 871/1753] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:03 2022] DELETE /identity/v3/users/901dda0ca7864044b68d1e5ddefbc0fd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:03.759341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:03.761760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:03.780222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b75724-2a02-4383-b143-6ca3bfd1cb59 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:03.848704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:03.849579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:03.849855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:03.850122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] PATH_INFO: `/v3/projects/5715dffef7d646c3a72a896a643c6e56` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:03.851316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5715dffef7d646c3a72a896a643c6e56)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:03.853369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:03.908690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5715dffef7d646c3a72a896a643c6e56', 'request_id': 'req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.917017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5715dffef7d646c3a72a896a643c6e56', 'request_id': 'req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.923968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5715dffef7d646c3a72a896a643c6e56', 'request_id': 'req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.927573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:03.942579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] The token cache is being invalidate because project 5715dffef7d646c3a72a896a643c6e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:03.942881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.943572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f2d5352a-b2f4-4f7b-9800-dbbb6727b47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:03.945145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 883/1754] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:03 2022] DELETE /identity/v3/projects/5715dffef7d646c3a72a896a643c6e56 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:03.950670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9cf6-2e13-4dee-bc05-17b785a6d596 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:03.950943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9cf6-2e13-4dee-bc05-17b785a6d596 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:03.951228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9cf6-2e13-4dee-bc05-17b785a6d596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:03.971604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-037b9cf6-2e13-4dee-bc05-17b785a6d596 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:03.972801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-037b9cf6-2e13-4dee-bc05-17b785a6d596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:04.024012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:04.039505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 872/1755] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:04.049679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de842e9a-38d8-42cf-bed2-69f3cfa04fd6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.049967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de842e9a-38d8-42cf-bed2-69f3cfa04fd6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.050243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de842e9a-38d8-42cf-bed2-69f3cfa04fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.072175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-de842e9a-38d8-42cf-bed2-69f3cfa04fd6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:04.074144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-de842e9a-38d8-42cf-bed2-69f3cfa04fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:04.082320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 884/1756] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:04.086978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9cf6-2e13-4dee-bc05-17b785a6d596 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.097577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9cf6-2e13-4dee-bc05-17b785a6d596 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.110233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.116156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.118545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.118545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.118545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:04.118545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.120268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.144696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de842e9a-38d8-42cf-bed2-69f3cfa04fd6 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.152298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 873/1757] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:04.155110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de842e9a-38d8-42cf-bed2-69f3cfa04fd6 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.160560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.161499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.161982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.162451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.164056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.166365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.210232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 885/1758] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:04 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:04.216321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.226902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cf111-d49f-421c-86d9-d1cbf1be1987 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.303875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.304998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.305494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.305878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.307237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.309526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.368777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 874/1759] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:04 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:04.369750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.381096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c12f0-0ce6-48af-91fe-405f5dd56ec9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.386403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.387093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.387400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.387708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.389614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.390871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.408381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.409170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 886/1760] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:04.429148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24577b-76f1-4111-924d-0c76cc8dfc04 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.443842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcd88f-7a7d-478a-8d57-528b8d7cdc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.444550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcd88f-7a7d-478a-8d57-528b8d7cdc12 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.444835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcd88f-7a7d-478a-8d57-528b8d7cdc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.445115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcd88f-7a7d-478a-8d57-528b8d7cdc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.445865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcd88f-7a7d-478a-8d57-528b8d7cdc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.447698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcd88f-7a7d-478a-8d57-528b8d7cdc12 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.458682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 875/1761] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:04.463354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.477096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3df99-09ad-400e-916b-b23d660034cd admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.484612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.485986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.486443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.486765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] PATH_INFO: `/v3/projects/cda0ddccf0344065b6e65100fcd8d31a/users/3820903c34d545118e475f4525ceeea5/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.504365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda0ddccf0344065b6e65100fcd8d31a, user_id=3820903c34d545118e475f4525ceeea5, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.507024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.523331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 887/1762] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:04 2022] PUT /identity/v3/projects/cda0ddccf0344065b6e65100fcd8d31a/users/3820903c34d545118e475f4525ceeea5/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:04.537654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56da00-2fcb-4a5c-9f8c-a4771518fe37 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.538015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56da00-2fcb-4a5c-9f8c-a4771518fe37 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.538287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56da00-2fcb-4a5c-9f8c-a4771518fe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.584913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-5c56da00-2fcb-4a5c-9f8c-a4771518fe37 None None] MFA Rules not processed for user `3820903c34d545118e475f4525ceeea5`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:04.586137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5c56da00-2fcb-4a5c-9f8c-a4771518fe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:04.674268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 876/1763] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:04.678647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.767166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfde505-a4fd-4d48-acc2-192daf14ff3d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.772664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.773431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.773783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.774122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.774906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.777331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.833566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 888/1764] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:04 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:04.833891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56da00-2fcb-4a5c-9f8c-a4771518fe37 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:04.860213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56da00-2fcb-4a5c-9f8c-a4771518fe37 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:04.937990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:04.956760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:04.957786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:04.958250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:04.958732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:04.960217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:04.962387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:04.992239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 877/1765] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:04 2022] POST /identity/v3/users => generated 300 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:04.996326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:05.016788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e468a-8c4c-495a-9c00-8b39e499674e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:05.036295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:05.037098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:05.037455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:05.037820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:05.038788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:05.040790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:05.055415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 889/1766] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:05.059166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:05.069542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f9ed5-bf51-4880-8e8f-e0d99bab224d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:05.075101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:05.076065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:05.076363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:05.076636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] PATH_INFO: `/v3/projects/bbbdbcae1dca42719c382bd1b827858d/users/a727747523784b08a6ab9e5033d70f68/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:05.101722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbbdbcae1dca42719c382bd1b827858d, user_id=a727747523784b08a6ab9e5033d70f68, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:05.105080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:05.130941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 878/1767] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:05 2022] PUT /identity/v3/projects/bbbdbcae1dca42719c382bd1b827858d/users/a727747523784b08a6ab9e5033d70f68/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:05.135100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:05.149375 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c125-bca5-421c-9d55-08eb38ff20b4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:05.252335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d1486-fc79-4129-925f-f97915781b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:05.253204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d1486-fc79-4129-925f-f97915781b5e admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:05.253686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d1486-fc79-4129-925f-f97915781b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:05.255015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d1486-fc79-4129-925f-f97915781b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:05.255969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d1486-fc79-4129-925f-f97915781b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:05.257683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d1486-fc79-4129-925f-f97915781b5e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:05.267621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:05.274864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 890/1768] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:05.286986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:05.298863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965689-07cf-4df4-8992-afb6a51c2d8b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:05.304177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:05.305119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:05.305404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:05.305686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] PATH_INFO: `/v3/projects/bbbdbcae1dca42719c382bd1b827858d/users/a727747523784b08a6ab9e5033d70f68/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:05.308169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbbdbcae1dca42719c382bd1b827858d, user_id=a727747523784b08a6ab9e5033d70f68, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:05.310879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:05.324197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 879/1769] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:05 2022] PUT /identity/v3/projects/bbbdbcae1dca42719c382bd1b827858d/users/a727747523784b08a6ab9e5033d70f68/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:05.337697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75eb32-e797-4099-84c9-391a06a78761 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:05.338132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75eb32-e797-4099-84c9-391a06a78761 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:05.338707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75eb32-e797-4099-84c9-391a06a78761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:05.368173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-2c75eb32-e797-4099-84c9-391a06a78761 None None] MFA Rules not processed for user `a727747523784b08a6ab9e5033d70f68`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:05.369576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2c75eb32-e797-4099-84c9-391a06a78761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:05.461441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 891/1770] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:05.475944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:05.478882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:05.563030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285a1a-fe8d-4b68-80fd-dfb1195c66b1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:05.567017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79e43a-f820-4006-b724-7bbf0b4fea1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:05.567803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79e43a-f820-4006-b724-7bbf0b4fea1a service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:05.568205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79e43a-f820-4006-b724-7bbf0b4fea1a service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:05.568965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79e43a-f820-4006-b724-7bbf0b4fea1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:05.581412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79e43a-f820-4006-b724-7bbf0b4fea1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:05.584275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79e43a-f820-4006-b724-7bbf0b4fea1a service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:05.598758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 880/1771] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:06.399831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75eb32-e797-4099-84c9-391a06a78761 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:06.538325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75eb32-e797-4099-84c9-391a06a78761 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:06.546804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:06.548081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:06.548494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:06.548781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] PATH_INFO: `/v3/users/3820903c34d545118e475f4525ceeea5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:06.550116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3820903c34d545118e475f4525ceeea5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:06.556514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:06.624524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3820903c34d545118e475f4525ceeea5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:06.642521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3820903c34d545118e475f4525ceeea5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:06.653966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3820903c34d545118e475f4525ceeea5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:06.663518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3820903c34d545118e475f4525ceeea5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:06.664722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3820903c34d545118e475f4525ceeea5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:06.675469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 892/1772] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:06 2022] DELETE /identity/v3/users/3820903c34d545118e475f4525ceeea5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:06.676963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:06.677458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:06.677635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:06.695607 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:06.697099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:06.767597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 881/1773] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:06.777623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:06.780506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:06.788469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:06.868258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f6249-d75d-4dcb-af55-6453cec85ad0 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:06.872502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:06.873590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:06.873931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:06.874280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:06.886563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:06.888069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:06.899792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 893/1774] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:07.269692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:07.271134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:07.302837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7108e1-2f26-4edb-83ee-16e35b2d2594 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:07.307857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:07.308776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:07.309062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:07.309332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] PATH_INFO: `/v3/projects/cda0ddccf0344065b6e65100fcd8d31a` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:07.312131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda0ddccf0344065b6e65100fcd8d31a)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:07.314055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:07.371560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:07.383412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda0ddccf0344065b6e65100fcd8d31a', 'request_id': 'req-24a73719-a3aa-4725-8653-f42cd44a148e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.387088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda0ddccf0344065b6e65100fcd8d31a', 'request_id': 'req-24a73719-a3aa-4725-8653-f42cd44a148e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.396277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda0ddccf0344065b6e65100fcd8d31a', 'request_id': 'req-24a73719-a3aa-4725-8653-f42cd44a148e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.415158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] The token cache is being invalidate because project cda0ddccf0344065b6e65100fcd8d31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:07.415363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.416025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.417659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 882/1775] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:07 2022] DELETE /identity/v3/projects/cda0ddccf0344065b6e65100fcd8d31a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:07.421710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:07.431821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea3574-d02a-4daa-9b26-f9669b60558f service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:07.499259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:07.500222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:07.501229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:07.501229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] PATH_INFO: `/v3/users/a727747523784b08a6ab9e5033d70f68` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:07.502195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a727747523784b08a6ab9e5033d70f68)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:07.504153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:07.556974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a727747523784b08a6ab9e5033d70f68'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.570082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a727747523784b08a6ab9e5033d70f68'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.580009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a727747523784b08a6ab9e5033d70f68'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.591462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a727747523784b08a6ab9e5033d70f68'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.591462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0d3d272a-60a4-42de-b193-60cda159df30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a727747523784b08a6ab9e5033d70f68'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:07.597327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 894/1776] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:07 2022] DELETE /identity/v3/users/a727747523784b08a6ab9e5033d70f68 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:08.051533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.053091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:08.067645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a73719-a3aa-4725-8653-f42cd44a148e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.155620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.156531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.156821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.157099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] PATH_INFO: `/v3/projects/bbbdbcae1dca42719c382bd1b827858d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.158322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbbdbcae1dca42719c382bd1b827858d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:08.160318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:08.215284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbbdbcae1dca42719c382bd1b827858d', 'request_id': 'req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:08.222773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbbdbcae1dca42719c382bd1b827858d', 'request_id': 'req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:08.228865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbbdbcae1dca42719c382bd1b827858d', 'request_id': 'req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:08.241468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] The token cache is being invalidate because project bbbdbcae1dca42719c382bd1b827858d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:08.241761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:08.242387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e6129347-2e44-45aa-b0d3-a69bbfcbd4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:08.243989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 883/1777] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:08 2022] DELETE /identity/v3/projects/bbbdbcae1dca42719c382bd1b827858d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:08.249263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bca9bd-eefb-4b69-b266-4b6a7b90df02 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.249584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bca9bd-eefb-4b69-b266-4b6a7b90df02 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.249867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bca9bd-eefb-4b69-b266-4b6a7b90df02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.268094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-37bca9bd-eefb-4b69-b266-4b6a7b90df02 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:08.269321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-37bca9bd-eefb-4b69-b266-4b6a7b90df02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:08.291796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:08.332857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 895/1778] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:08.341889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f4dc7-a2ba-4b55-a5e4-044da0073eae None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.342199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f4dc7-a2ba-4b55-a5e4-044da0073eae None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.342534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f4dc7-a2ba-4b55-a5e4-044da0073eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.363141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-487f4dc7-a2ba-4b55-a5e4-044da0073eae None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:08.364410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-487f4dc7-a2ba-4b55-a5e4-044da0073eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:08.373606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 884/1779] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:08.378158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bca9bd-eefb-4b69-b266-4b6a7b90df02 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.387972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bca9bd-eefb-4b69-b266-4b6a7b90df02 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.393198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.393870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.394179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.394459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.395199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:08.395556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:08.397170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:08.409854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 896/1780] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:08.414258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f4dc7-a2ba-4b55-a5e4-044da0073eae None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.424491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f4dc7-a2ba-4b55-a5e4-044da0073eae None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.430063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.430726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.431013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.431289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.432194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:08.434188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:08.463135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 885/1781] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:08 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:08.468087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.478324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51310949-6cdf-4cd9-9e73-18aa03665854 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.542187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.542871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.543157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.543465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.544277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:08.546290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:08.572075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 897/1782] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:08 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:08.577240 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.587651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd60941-31f2-49e2-8320-ab419e9d0e96 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.593165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.593842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.594127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.594403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.595139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:08.596794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:08.608112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 886/1783] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:08.612295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.624860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996fa60-fa4a-4cdd-b323-822d433af943 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.631618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a914ab-154e-4b56-8b5b-7ffe137a72bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.632880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a914ab-154e-4b56-8b5b-7ffe137a72bb admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.633104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a914ab-154e-4b56-8b5b-7ffe137a72bb admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.633445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a914ab-154e-4b56-8b5b-7ffe137a72bb admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.634313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a914ab-154e-4b56-8b5b-7ffe137a72bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:08.636575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a914ab-154e-4b56-8b5b-7ffe137a72bb admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:08.652618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 898/1784] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:08.656944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.667827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c59c25-ae79-4204-a4eb-27ad311f2e1a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.672558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.673587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.673935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.674284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] PATH_INFO: `/v3/projects/c95ed362b3bc4725abea964176ff19b9/users/ce9baecbe1ee469a9a424d4d93e9e53c/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.690163 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95ed362b3bc4725abea964176ff19b9, user_id=ce9baecbe1ee469a9a424d4d93e9e53c, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:08.692961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:08.695289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:08.709287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 887/1785] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:08 2022] PUT /identity/v3/projects/c95ed362b3bc4725abea964176ff19b9/users/ce9baecbe1ee469a9a424d4d93e9e53c/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:08.721947 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2441080-b54e-4202-b9c9-7382fb8b7752 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.722277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2441080-b54e-4202-b9c9-7382fb8b7752 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.722566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2441080-b54e-4202-b9c9-7382fb8b7752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.750395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f2441080-b54e-4202-b9c9-7382fb8b7752 None None] MFA Rules not processed for user `ce9baecbe1ee469a9a424d4d93e9e53c`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:08.751792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f2441080-b54e-4202-b9c9-7382fb8b7752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:08.848205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 899/1786] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:08.853336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:08.973861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f59bd1-5235-406c-b15c-dcad68c9186c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:08.987387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:08.988171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:08.988568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:08.989022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:08.989859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:09.001062 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:09.043440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 888/1787] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:08 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:09.046639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2441080-b54e-4202-b9c9-7382fb8b7752 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:09.059236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2441080-b54e-4202-b9c9-7382fb8b7752 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:09.081550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:09.174162 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:09.175174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:09.175765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:09.176183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:09.178368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:09.180780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:09.212007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 900/1788] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:09 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:09.216631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:09.231090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca63e54-9281-42da-be8c-7b96f19a8fe8 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:09.236179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:09.236880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:09.237165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:09.237441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:09.238191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:09.240095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:09.252641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 889/1789] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:09.256794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:09.268215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf838-f440-4101-b749-b9671822d332 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:09.273247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:09.274205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:09.274486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:09.274761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] PATH_INFO: `/v3/projects/301bb81b756f46a0b1bdfe93bdac44c6/users/2c2aeaa1cd9d48ea84a25465a7afba30/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:09.295886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301bb81b756f46a0b1bdfe93bdac44c6, user_id=2c2aeaa1cd9d48ea84a25465a7afba30, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:09.298714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:09.315988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 901/1790] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:09 2022] PUT /identity/v3/projects/301bb81b756f46a0b1bdfe93bdac44c6/users/2c2aeaa1cd9d48ea84a25465a7afba30/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:09.320049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:09.330912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd89f3-dc7c-47e6-aeda-beaba7ce2fca admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:09.433156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3b45-939c-40ad-89c4-d26234a245fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:09.434135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3b45-939c-40ad-89c4-d26234a245fe admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:09.434726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3b45-939c-40ad-89c4-d26234a245fe admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:09.435957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3b45-939c-40ad-89c4-d26234a245fe admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:09.436999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a3b45-939c-40ad-89c4-d26234a245fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:09.444084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a3b45-939c-40ad-89c4-d26234a245fe admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:09.461630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 890/1791] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:09.467563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:09.479076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936e7f6-9a4e-4fac-982a-fa4c93036575 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:09.488070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:09.489658 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:09.491856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:09.492164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] PATH_INFO: `/v3/projects/301bb81b756f46a0b1bdfe93bdac44c6/users/2c2aeaa1cd9d48ea84a25465a7afba30/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:09.506575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301bb81b756f46a0b1bdfe93bdac44c6, user_id=2c2aeaa1cd9d48ea84a25465a7afba30, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:09.512099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:09.561610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 902/1792] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:09 2022] PUT /identity/v3/projects/301bb81b756f46a0b1bdfe93bdac44c6/users/2c2aeaa1cd9d48ea84a25465a7afba30/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:09.571854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9c50-302d-4f03-a85c-f491359a8026 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:09.572790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9c50-302d-4f03-a85c-f491359a8026 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:09.573419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9c50-302d-4f03-a85c-f491359a8026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:09.630189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c72a9c50-302d-4f03-a85c-f491359a8026 None None] MFA Rules not processed for user `2c2aeaa1cd9d48ea84a25465a7afba30`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:09.632396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c72a9c50-302d-4f03-a85c-f491359a8026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:09.731712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:09.762589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 891/1793] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:09.776019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:09.777918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:09.975457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7da91a-3221-4dce-a626-501c167012b4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:09.984512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438c4bb-bed7-4f13-8332-d7006257d3f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:09.984928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438c4bb-bed7-4f13-8332-d7006257d3f8 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:09.985219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438c4bb-bed7-4f13-8332-d7006257d3f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:09.985498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438c4bb-bed7-4f13-8332-d7006257d3f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:10.115269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438c4bb-bed7-4f13-8332-d7006257d3f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:10.117271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438c4bb-bed7-4f13-8332-d7006257d3f8 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:10.128275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 903/1794] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:10.581828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da70487-0eaa-4474-b6b1-eb941237b23d None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:10.584515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da70487-0eaa-4474-b6b1-eb941237b23d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:10.585110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da70487-0eaa-4474-b6b1-eb941237b23d None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:10.590223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 892/1795] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:52:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:52:10.624862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:10.625378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:10.625458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:10.694236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:10.697926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:10.808147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:10.915987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 904/1796] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:52:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:11.209712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da70487-0eaa-4474-b6b1-eb941237b23d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:11.212770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4da70487-0eaa-4474-b6b1-eb941237b23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:11.238065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da70487-0eaa-4474-b6b1-eb941237b23d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:11.247180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f840485-d5f6-4294-bc85-d4117f8cf442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:11.248515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840485-d5f6-4294-bc85-d4117f8cf442 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:11.248803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840485-d5f6-4294-bc85-d4117f8cf442 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:11.249080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840485-d5f6-4294-bc85-d4117f8cf442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:11.261798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f840485-d5f6-4294-bc85-d4117f8cf442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:11.263537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f840485-d5f6-4294-bc85-d4117f8cf442 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:11.276716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 893/1797] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:11.424298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:11.427831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:11.518506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd0a97-103b-4f63-9e92-73f3a4f2edbf None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:11.525701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:11.527350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:11.527730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:11.528074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] PATH_INFO: `/v3/users/b39f809d6ee343e9b8c90083d5f37dc6` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:11.529527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b39f809d6ee343e9b8c90083d5f37dc6)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:11.531649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:11.590796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b39f809d6ee343e9b8c90083d5f37dc6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:11.609294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b39f809d6ee343e9b8c90083d5f37dc6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:11.620157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b39f809d6ee343e9b8c90083d5f37dc6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:11.627871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b39f809d6ee343e9b8c90083d5f37dc6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:11.628808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b39f809d6ee343e9b8c90083d5f37dc6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:11.635857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 905/1798] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:11 2022] DELETE /identity/v3/users/b39f809d6ee343e9b8c90083d5f37dc6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:11.641247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:11.641745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:11.641863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:11.660185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:11.661393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:11.723941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 894/1799] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:11.734218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:11.736017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:11.782605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:11.798877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0f8ab-7a8c-4dae-9a4f-7909cdfea091 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:11.802699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:11.803351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:11.803664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:11.803963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:11.815251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:11.816782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:11.827457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 906/1800] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:12.174972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:12.176512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:12.191187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2f1fb-7fa3-46c6-a4d7-e50d339e26f0 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:12.216725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:12.218037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:12.218527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:12.219229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] PATH_INFO: `/v3/projects/2203638a5cdb44e7b427ae4c15244d2e` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:12.222566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2203638a5cdb44e7b427ae4c15244d2e)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:12.228316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:12.300732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2203638a5cdb44e7b427ae4c15244d2e', 'request_id': 'req-8a01fc62-06c2-42b2-857d-bdc094768dd1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.309591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2203638a5cdb44e7b427ae4c15244d2e', 'request_id': 'req-8a01fc62-06c2-42b2-857d-bdc094768dd1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.316864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2203638a5cdb44e7b427ae4c15244d2e', 'request_id': 'req-8a01fc62-06c2-42b2-857d-bdc094768dd1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.340617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] The token cache is being invalidate because project 2203638a5cdb44e7b427ae4c15244d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:12.341398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.344830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.347499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 895/1801] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:12 2022] DELETE /identity/v3/projects/2203638a5cdb44e7b427ae4c15244d2e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:12.351299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:12.363789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f93312-a690-4fac-aad5-55d8a53a675f service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:12.462983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:12.465101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:12.465384 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:12.465677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] PATH_INFO: `/v3/users/2c9a9e752cba460aabaeac31519db01e` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:12.467295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c9a9e752cba460aabaeac31519db01e)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:12.469324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:12.590058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c9a9e752cba460aabaeac31519db01e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.622770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c9a9e752cba460aabaeac31519db01e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.638673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c9a9e752cba460aabaeac31519db01e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.647883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c9a9e752cba460aabaeac31519db01e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.648757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-82364b33-5013-467d-9b86-f8a0a2db4456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c9a9e752cba460aabaeac31519db01e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:12.672341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 907/1802] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:12 2022] DELETE /identity/v3/users/2c9a9e752cba460aabaeac31519db01e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:13.433437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:13.434939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:13.461170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01fc62-06c2-42b2-857d-bdc094768dd1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:13.565224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:13.566140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:13.566424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:13.566703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] PATH_INFO: `/v3/projects/e84b22d3fb0f49e98399223fa3c69195` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:13.568915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84b22d3fb0f49e98399223fa3c69195)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:13.572048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:13.575587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:13.657313 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84b22d3fb0f49e98399223fa3c69195', 'request_id': 'req-d308537d-b9a4-4741-8939-7bad946b073c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:13.665244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84b22d3fb0f49e98399223fa3c69195', 'request_id': 'req-d308537d-b9a4-4741-8939-7bad946b073c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:13.671470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84b22d3fb0f49e98399223fa3c69195', 'request_id': 'req-d308537d-b9a4-4741-8939-7bad946b073c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:13.695673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] The token cache is being invalidate because project e84b22d3fb0f49e98399223fa3c69195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:13.695984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:13.696593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d308537d-b9a4-4741-8939-7bad946b073c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:13.698737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 896/1803] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:13 2022] DELETE /identity/v3/projects/e84b22d3fb0f49e98399223fa3c69195 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:13.705273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471f945-8593-4e5e-bb06-0336b554f3ec None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:13.705716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471f945-8593-4e5e-bb06-0336b554f3ec None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:13.705951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471f945-8593-4e5e-bb06-0336b554f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:13.727632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8471f945-8593-4e5e-bb06-0336b554f3ec None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:13.728630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8471f945-8593-4e5e-bb06-0336b554f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:13.797883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 908/1804] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:13.805901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69507bba-bddf-497d-9a0b-1c5763da469f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:13.806196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69507bba-bddf-497d-9a0b-1c5763da469f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:13.806472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69507bba-bddf-497d-9a0b-1c5763da469f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:13.826564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-69507bba-bddf-497d-9a0b-1c5763da469f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:13.827785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-69507bba-bddf-497d-9a0b-1c5763da469f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:13.836777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 897/1805] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:13.841939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471f945-8593-4e5e-bb06-0336b554f3ec None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:13.847361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:13.853542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471f945-8593-4e5e-bb06-0336b554f3ec None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:13.863129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:13.864013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:13.864617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:13.865814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:13.866770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:13.867418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:13.869202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:13.883623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 909/1806] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:52:13.888235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69507bba-bddf-497d-9a0b-1c5763da469f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:13.898355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69507bba-bddf-497d-9a0b-1c5763da469f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:13.904668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:13.905478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:13.905878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:13.906329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:13.907255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:13.909652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:13.937445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 898/1807] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:13 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:13.944434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:13.959561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21ecc4-1960-4af7-bb54-053e0f66cd89 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.020711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.021395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.021680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.021960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.022663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.024730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.048893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 910/1808] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:13 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:14.050084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.060677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24415a57-4f6b-41db-aadc-6bf20723f70b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.065679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.066389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.066651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.066952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.067709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.069320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.080380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 899/1809] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:14.084441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.096359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf2fbb-b729-421b-9fbc-08b8870fd259 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.101200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1e8ee-775b-4975-b74f-90e03a03a012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.101865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1e8ee-775b-4975-b74f-90e03a03a012 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.102144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1e8ee-775b-4975-b74f-90e03a03a012 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.102422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1e8ee-775b-4975-b74f-90e03a03a012 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.103155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1e8ee-775b-4975-b74f-90e03a03a012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.105499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1e8ee-775b-4975-b74f-90e03a03a012 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.119789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 911/1810] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:14.124170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.134640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9c082-67e5-4625-8ac9-6593898cdb78 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.139345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.140298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.140584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.140862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] PATH_INFO: `/v3/projects/7cc05913634e41b7a85dd7127ca4ab7e/users/6a29985274f24fa0b0f0beff3b9ec614/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.155423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc05913634e41b7a85dd7127ca4ab7e, user_id=6a29985274f24fa0b0f0beff3b9ec614, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.158112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.175016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 900/1811] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:14 2022] PUT /identity/v3/projects/7cc05913634e41b7a85dd7127ca4ab7e/users/6a29985274f24fa0b0f0beff3b9ec614/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:14.185574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20dd01-6a48-47e5-9ca2-5c2276340acf None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.185863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20dd01-6a48-47e5-9ca2-5c2276340acf None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.186314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20dd01-6a48-47e5-9ca2-5c2276340acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.213307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-be20dd01-6a48-47e5-9ca2-5c2276340acf None None] MFA Rules not processed for user `6a29985274f24fa0b0f0beff3b9ec614`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:14.214934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-be20dd01-6a48-47e5-9ca2-5c2276340acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:14.300723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 912/1812] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:14.305544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.394197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad831bda-3097-4c65-aedf-8f0b473e9175 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.398832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.399557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.399843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.400118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.400817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.402813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.428967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 901/1813] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:14 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:14.433476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20dd01-6a48-47e5-9ca2-5c2276340acf None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.449004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20dd01-6a48-47e5-9ca2-5c2276340acf None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.504123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:14.511470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.512315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.512600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.512878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.513590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.515600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.538620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 913/1814] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:14 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:14.539820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.550462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f59b-1d5b-4351-bc4c-56fd585ee765 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.556458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.557278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.557612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.558821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.559848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.562066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.569932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:14.579651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 902/1815] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:14.584793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.596723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f617609-f371-487d-8e61-7e946bed9051 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.601695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.602634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.602919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.603196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] PATH_INFO: `/v3/projects/bcea5345dcfc4feb9d9bbe7cf7df28f1/users/af991f74f5a646438eb27c2f07dd6ff7/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.619299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcea5345dcfc4feb9d9bbe7cf7df28f1, user_id=af991f74f5a646438eb27c2f07dd6ff7, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.622367 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.636432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 914/1816] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:14 2022] PUT /identity/v3/projects/bcea5345dcfc4feb9d9bbe7cf7df28f1/users/af991f74f5a646438eb27c2f07dd6ff7/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:14.641660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.655763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e26b4-3a1f-482e-a2d9-cd4c2f533d59 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.721971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59de88-c503-4f76-9447-cf927251e828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.722667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59de88-c503-4f76-9447-cf927251e828 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.723280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59de88-c503-4f76-9447-cf927251e828 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.723598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59de88-c503-4f76-9447-cf927251e828 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.724368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59de88-c503-4f76-9447-cf927251e828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.725974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59de88-c503-4f76-9447-cf927251e828 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.737780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 903/1817] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:14.742087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.753336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe5ba9-1f73-42e5-b295-4c831a15e949 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.758295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.759194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.759508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.759783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] PATH_INFO: `/v3/projects/bcea5345dcfc4feb9d9bbe7cf7df28f1/users/af991f74f5a646438eb27c2f07dd6ff7/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.762516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcea5345dcfc4feb9d9bbe7cf7df28f1, user_id=af991f74f5a646438eb27c2f07dd6ff7, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:14.765183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:14.776293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 915/1818] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:14 2022] PUT /identity/v3/projects/bcea5345dcfc4feb9d9bbe7cf7df28f1/users/af991f74f5a646438eb27c2f07dd6ff7/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:14.787599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba86f32-f26b-4e3c-8a5a-af9589468342 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.788274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba86f32-f26b-4e3c-8a5a-af9589468342 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.788785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba86f32-f26b-4e3c-8a5a-af9589468342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:14.828954 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-aba86f32-f26b-4e3c-8a5a-af9589468342 None None] MFA Rules not processed for user `af991f74f5a646438eb27c2f07dd6ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:14.830287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-aba86f32-f26b-4e3c-8a5a-af9589468342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:14.903259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 904/1819] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:14.916316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:14.918749 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:14.987037 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795075d-9afb-4221-9351-824dca5dcee6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:14.993005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:14.994273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:14.995286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:14.995798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:15.008739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:15.011830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:15.022995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 916/1820] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:16.325152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba86f32-f26b-4e3c-8a5a-af9589468342 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:16.360430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba86f32-f26b-4e3c-8a5a-af9589468342 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:16.372778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c2af2-4779-4914-b7c2-13e0ea5d3df1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:16.374315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2af2-4779-4914-b7c2-13e0ea5d3df1 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:16.374608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2af2-4779-4914-b7c2-13e0ea5d3df1 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:16.374888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2af2-4779-4914-b7c2-13e0ea5d3df1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:16.381074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ec1c2af2-4779-4914-b7c2-13e0ea5d3df1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:16.479010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c2af2-4779-4914-b7c2-13e0ea5d3df1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:16.480677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c2af2-4779-4914-b7c2-13e0ea5d3df1 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:16.495230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 905/1821] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:16.825452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:16.942483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3702c-e15f-4dec-b83d-15d121971c7c service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:16.951680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:16.952751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:16.953140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:16.953561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] PATH_INFO: `/v3/users/6a29985274f24fa0b0f0beff3b9ec614` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:16.955808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a29985274f24fa0b0f0beff3b9ec614)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:16.957958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:17.025008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a29985274f24fa0b0f0beff3b9ec614'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:17.033866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:17.049402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a29985274f24fa0b0f0beff3b9ec614'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:17.059816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a29985274f24fa0b0f0beff3b9ec614'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:17.068003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a29985274f24fa0b0f0beff3b9ec614'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:17.068810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a29985274f24fa0b0f0beff3b9ec614'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:17.076573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 917/1822] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:16 2022] DELETE /identity/v3/users/6a29985274f24fa0b0f0beff3b9ec614 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:17.081021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:17.081629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:17.081750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:17.122361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:17.123729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:17.138794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:17.207396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 906/1823] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:17.226452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:17.228272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:17.310497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fead37-2baa-48ba-bd65-22d43a82485d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:17.315721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:17.316633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:17.317028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:17.317403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:17.328902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:17.330583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:17.342937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 918/1824] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:17.960770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:17.965874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:17.990342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac875e-592f-445c-ac46-98a9b61d97a4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:18.000277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:18.018155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:18.018760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:18.019231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] PATH_INFO: `/v3/projects/7cc05913634e41b7a85dd7127ca4ab7e` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:18.021808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc05913634e41b7a85dd7127ca4ab7e)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:18.030934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:18.095050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc05913634e41b7a85dd7127ca4ab7e', 'request_id': 'req-abab54e8-2369-49da-8b91-f56f9dda5257'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.102335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc05913634e41b7a85dd7127ca4ab7e', 'request_id': 'req-abab54e8-2369-49da-8b91-f56f9dda5257'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.108440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc05913634e41b7a85dd7127ca4ab7e', 'request_id': 'req-abab54e8-2369-49da-8b91-f56f9dda5257'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.132083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] The token cache is being invalidate because project 7cc05913634e41b7a85dd7127ca4ab7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:18.132405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.133038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.134650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 907/1825] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:17 2022] DELETE /identity/v3/projects/7cc05913634e41b7a85dd7127ca4ab7e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:18.138674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:18.148697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442249-3af4-449b-854c-108009d36d06 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:18.207632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:18.208549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:18.208820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:18.209092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] PATH_INFO: `/v3/users/af991f74f5a646438eb27c2f07dd6ff7` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:18.210303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af991f74f5a646438eb27c2f07dd6ff7)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:18.212312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:18.263774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af991f74f5a646438eb27c2f07dd6ff7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.279571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af991f74f5a646438eb27c2f07dd6ff7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.289475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af991f74f5a646438eb27c2f07dd6ff7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.296273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:18.300475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af991f74f5a646438eb27c2f07dd6ff7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.301603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-634f84ea-9eef-4986-9a79-fcdcd022b570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af991f74f5a646438eb27c2f07dd6ff7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:18.319812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 919/1826] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:18 2022] DELETE /identity/v3/users/af991f74f5a646438eb27c2f07dd6ff7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:19.385255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:19.387141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:19.400384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab54e8-2369-49da-8b91-f56f9dda5257 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:19.465458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:19.466344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:19.466989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:19.467270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] PATH_INFO: `/v3/projects/bcea5345dcfc4feb9d9bbe7cf7df28f1` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:19.468510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcea5345dcfc4feb9d9bbe7cf7df28f1)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:19.470442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:19.515138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:19.525183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcea5345dcfc4feb9d9bbe7cf7df28f1', 'request_id': 'req-03f8496f-5eb7-4e0f-b0de-f462a580cd12'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:19.531753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcea5345dcfc4feb9d9bbe7cf7df28f1', 'request_id': 'req-03f8496f-5eb7-4e0f-b0de-f462a580cd12'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:19.537635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcea5345dcfc4feb9d9bbe7cf7df28f1', 'request_id': 'req-03f8496f-5eb7-4e0f-b0de-f462a580cd12'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:19.552087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] The token cache is being invalidate because project bcea5345dcfc4feb9d9bbe7cf7df28f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:19.552602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:19.553301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-03f8496f-5eb7-4e0f-b0de-f462a580cd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:19.559402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 908/1827] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:19 2022] DELETE /identity/v3/projects/bcea5345dcfc4feb9d9bbe7cf7df28f1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:19.561221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49aeedf-ca39-420e-8bb7-7580bcaabbd1 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:19.561696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49aeedf-ca39-420e-8bb7-7580bcaabbd1 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:19.561774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49aeedf-ca39-420e-8bb7-7580bcaabbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:19.582209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d49aeedf-ca39-420e-8bb7-7580bcaabbd1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:19.583532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d49aeedf-ca39-420e-8bb7-7580bcaabbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:19.682989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 920/1828] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:19.690877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e958-cf4f-47de-8ebc-f686b8795111 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:19.691105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e958-cf4f-47de-8ebc-f686b8795111 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:19.691417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e958-cf4f-47de-8ebc-f686b8795111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:19.722881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b583e958-cf4f-47de-8ebc-f686b8795111 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:19.725640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b583e958-cf4f-47de-8ebc-f686b8795111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:19.736696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 909/1829] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:19.741451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49aeedf-ca39-420e-8bb7-7580bcaabbd1 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:19.753765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49aeedf-ca39-420e-8bb7-7580bcaabbd1 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:19.759732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:19.760484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:19.760833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:19.761732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:19.762554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:19.762811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:19.764473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:19.778471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 921/1830] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:19.782903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583e958-cf4f-47de-8ebc-f686b8795111 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:19.793412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583e958-cf4f-47de-8ebc-f686b8795111 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:19.800087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:19.800793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:19.801532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:19.801699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:19.802566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:19.804615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:19.831713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 910/1831] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:19 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:19.835853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:19.847089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9104e-4f3b-4de4-97f8-52a2347464f5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:19.911786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:19.912760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:19.913121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:19.913372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:19.914297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:19.916301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:19.941512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 922/1832] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:19 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:19.942944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:19.953928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad8a4b-4c05-4e6c-a7ee-f7628003db17 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:19.960649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:19.961401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:19.961753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:19.962099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:19.962969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:19.965351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:19.977818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 911/1833] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:19.984829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:19.996478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa9f43-f4a4-43fa-a9fc-5f5db14aa749 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.001844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84864bed-6975-4dbb-9288-24d399febfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.002746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84864bed-6975-4dbb-9288-24d399febfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.003038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84864bed-6975-4dbb-9288-24d399febfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.003322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84864bed-6975-4dbb-9288-24d399febfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.004105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84864bed-6975-4dbb-9288-24d399febfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.005714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84864bed-6975-4dbb-9288-24d399febfe3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.021431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 923/1834] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:20.023028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.034724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab345b16-66fc-4a84-bbec-7e6e78e3e212 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.046988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.048305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.048462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.048780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] PATH_INFO: `/v3/projects/fbe3d9b5171e48a798ce9b7934849c36/users/2708fb812d34411181fcc281608b3b46/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.065879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe3d9b5171e48a798ce9b7934849c36, user_id=2708fb812d34411181fcc281608b3b46, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.068751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.089959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 912/1835] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:20 2022] PUT /identity/v3/projects/fbe3d9b5171e48a798ce9b7934849c36/users/2708fb812d34411181fcc281608b3b46/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:20.101551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ea919-dca3-471a-95ce-f3d29d026757 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.102103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ea919-dca3-471a-95ce-f3d29d026757 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.102386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ea919-dca3-471a-95ce-f3d29d026757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.151996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8e0ea919-dca3-471a-95ce-f3d29d026757 None None] MFA Rules not processed for user `2708fb812d34411181fcc281608b3b46`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:20.153365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8e0ea919-dca3-471a-95ce-f3d29d026757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:20.219165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:20.243282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 924/1836] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:20.244734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.316985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da574ac1-2417-424b-8a5b-ffc53c5def44 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.322567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.323443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.325115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.325115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.325115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.326793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.366987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 913/1837] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:20 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:20.367214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ea919-dca3-471a-95ce-f3d29d026757 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.382839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ea919-dca3-471a-95ce-f3d29d026757 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.465561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.466342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.467645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.467645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.468075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.473078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.520304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 925/1838] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:20 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:20.524476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.539670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06bf62-d2c8-4c11-b62b-8c5015589cb2 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.545469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.546198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.546540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.546823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.547923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.549631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.567420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.567959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 914/1839] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:20.579901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c812-6f88-4689-9b6f-323d680e62b9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.586199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.587269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.587663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.588012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] PATH_INFO: `/v3/projects/dc3d1cb72aee4b9881c3b40dcbfa3407/users/cc4cb76932814bf1964c520032c834e3/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.607478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3d1cb72aee4b9881c3b40dcbfa3407, user_id=cc4cb76932814bf1964c520032c834e3, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.610152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.628338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 926/1840] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:20 2022] PUT /identity/v3/projects/dc3d1cb72aee4b9881c3b40dcbfa3407/users/cc4cb76932814bf1964c520032c834e3/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:20.635560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.649080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa18f29-2bf0-4fce-905d-678cedf2b890 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.695345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:20.721404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdd7c3-49f2-4093-96d3-1d5d1911b32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.722469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdd7c3-49f2-4093-96d3-1d5d1911b32a admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.723019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdd7c3-49f2-4093-96d3-1d5d1911b32a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.723477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdd7c3-49f2-4093-96d3-1d5d1911b32a admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.724478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdd7c3-49f2-4093-96d3-1d5d1911b32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.726194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdd7c3-49f2-4093-96d3-1d5d1911b32a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.744984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 915/1841] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:20.746930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.759689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dc4b3-7e31-4fe3-be20-f3f8bf086b29 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:20.770216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:20.771684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.772059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.772401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] PATH_INFO: `/v3/projects/dc3d1cb72aee4b9881c3b40dcbfa3407/users/cc4cb76932814bf1964c520032c834e3/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.775221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3d1cb72aee4b9881c3b40dcbfa3407, user_id=cc4cb76932814bf1964c520032c834e3, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:20.778764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:20.791393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 927/1842] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:20 2022] PUT /identity/v3/projects/dc3d1cb72aee4b9881c3b40dcbfa3407/users/cc4cb76932814bf1964c520032c834e3/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:20.802023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3465-bb3c-492b-8462-73ab260483c8 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:20.802317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3465-bb3c-492b-8462-73ab260483c8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:20.803059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3465-bb3c-492b-8462-73ab260483c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:20.834729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c74e3465-bb3c-492b-8462-73ab260483c8 None None] MFA Rules not processed for user `cc4cb76932814bf1964c520032c834e3`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:20.836082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c74e3465-bb3c-492b-8462-73ab260483c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:20.923567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 916/1843] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:20.936008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:20.937926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:21.003064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a86cbe-7cad-4d1d-a492-3c7e909f1491 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:21.006870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ae51b-e0ee-4211-b26f-ad8dab143009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:21.007661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae51b-e0ee-4211-b26f-ad8dab143009 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:21.008015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae51b-e0ee-4211-b26f-ad8dab143009 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:21.008437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae51b-e0ee-4211-b26f-ad8dab143009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:21.020687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ae51b-e0ee-4211-b26f-ad8dab143009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:21.022804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ae51b-e0ee-4211-b26f-ad8dab143009 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:21.034248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 928/1844] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:21.131152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3465-bb3c-492b-8462-73ab260483c8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:21.263949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3465-bb3c-492b-8462-73ab260483c8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:21.271099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:21.272463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:21.272463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:21.272983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] PATH_INFO: `/v3/users/2708fb812d34411181fcc281608b3b46` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:21.274383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2708fb812d34411181fcc281608b3b46)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:21.282613 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:21.366768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2708fb812d34411181fcc281608b3b46'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:21.385850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:21.388386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2708fb812d34411181fcc281608b3b46'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:21.423456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2708fb812d34411181fcc281608b3b46'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:21.452244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2708fb812d34411181fcc281608b3b46'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:21.453700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2708fb812d34411181fcc281608b3b46'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:21.465695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 917/1845] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:21 2022] DELETE /identity/v3/users/2708fb812d34411181fcc281608b3b46 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:21.470069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da60b9-a073-45ed-a1e4-9c5069a89b16 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:21.470412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da60b9-a073-45ed-a1e4-9c5069a89b16 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:21.470698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da60b9-a073-45ed-a1e4-9c5069a89b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:21.493907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a8da60b9-a073-45ed-a1e4-9c5069a89b16 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:21.495143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a8da60b9-a073-45ed-a1e4-9c5069a89b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:21.557473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:21.603183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 929/1846] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:21.611819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:21.614029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:21.710293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385d69-1c6c-47f6-bf1a-46d1612c05ae None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:21.711080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385d69-1c6c-47f6-bf1a-46d1612c05ae None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:21.711080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385d69-1c6c-47f6-bf1a-46d1612c05ae None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:21.712592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 930/1847] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:52:21.740664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:21.740960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:21.741241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:21.801134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:21.802590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:21.971936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 931/1848] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:52:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:22.057260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c98c71-d5ae-4fcb-80b3-923f29a91419 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:22.063942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:22.064642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:22.064926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:22.065199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:22.081418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:22.083210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:22.105626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 918/1849] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:22.194789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:22.196384 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:22.220763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1227fec-cbb8-4c4d-b546-a2405db427f3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:22.235465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ea671-531e-414b-b829-53c6141732da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:22.236081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ea671-531e-414b-b829-53c6141732da service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:22.236390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ea671-531e-414b-b829-53c6141732da service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:22.236946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ea671-531e-414b-b829-53c6141732da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:22.252156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ea671-531e-414b-b829-53c6141732da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:22.254646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ea671-531e-414b-b829-53c6141732da service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:22.272964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 932/1850] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:22.536919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:22.538493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:22.552075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0c6b-4531-4dcf-b7c8-eb33568bb97b service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:22.557025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:22.558416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:22.558710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:22.558986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] PATH_INFO: `/v3/projects/fbe3d9b5171e48a798ce9b7934849c36` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:22.560249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbe3d9b5171e48a798ce9b7934849c36)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:22.562213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:22.613689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbe3d9b5171e48a798ce9b7934849c36', 'request_id': 'req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.620092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbe3d9b5171e48a798ce9b7934849c36', 'request_id': 'req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.625905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbe3d9b5171e48a798ce9b7934849c36', 'request_id': 'req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.640118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] The token cache is being invalidate because project fbe3d9b5171e48a798ce9b7934849c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:22.640409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.641087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.642668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 919/1851] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:22 2022] DELETE /identity/v3/projects/fbe3d9b5171e48a798ce9b7934849c36 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:22.646901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ea671-531e-414b-b829-53c6141732da service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:22.657473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ea671-531e-414b-b829-53c6141732da service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:22.715510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:22.716398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:22.717029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:22.717311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] PATH_INFO: `/v3/users/cc4cb76932814bf1964c520032c834e3` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:22.718505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4cb76932814bf1964c520032c834e3)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:22.720462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:22.769821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4cb76932814bf1964c520032c834e3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.782336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4cb76932814bf1964c520032c834e3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.784301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:22.794671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4cb76932814bf1964c520032c834e3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.801961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4cb76932814bf1964c520032c834e3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.802614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69a2212c-270a-4bac-a3e9-3594c98e7803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4cb76932814bf1964c520032c834e3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:22.808652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 933/1852] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:22 2022] DELETE /identity/v3/users/cc4cb76932814bf1964c520032c834e3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:23.113258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.115037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:23.136989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8b4e9-3a72-4b4c-82de-ac27c815b8d7 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:23.220029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:23.221256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.221738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.222496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] PATH_INFO: `/v3/projects/dc3d1cb72aee4b9881c3b40dcbfa3407` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.224375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3d1cb72aee4b9881c3b40dcbfa3407)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:23.227307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:23.237789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:23.290652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3d1cb72aee4b9881c3b40dcbfa3407', 'request_id': 'req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:23.298776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3d1cb72aee4b9881c3b40dcbfa3407', 'request_id': 'req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:23.307461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3d1cb72aee4b9881c3b40dcbfa3407', 'request_id': 'req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:23.326612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] The token cache is being invalidate because project dc3d1cb72aee4b9881c3b40dcbfa3407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:23.326882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:23.327576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6c2cedfe-0455-4f9c-8cba-0e5d1716751a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:23.332376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 920/1853] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:23 2022] DELETE /identity/v3/projects/dc3d1cb72aee4b9881c3b40dcbfa3407 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:23.337502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6f1bb-e98f-4067-aaad-5e210529795e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.337889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6f1bb-e98f-4067-aaad-5e210529795e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.338173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6f1bb-e98f-4067-aaad-5e210529795e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.365016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7ed6f1bb-e98f-4067-aaad-5e210529795e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:23.366333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7ed6f1bb-e98f-4067-aaad-5e210529795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:23.459440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 934/1854] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:23.470895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceea210-16d9-4eec-928b-4b59f2013f50 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.471185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceea210-16d9-4eec-928b-4b59f2013f50 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.471495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceea210-16d9-4eec-928b-4b59f2013f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.489829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-7ceea210-16d9-4eec-928b-4b59f2013f50 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:23.491229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7ceea210-16d9-4eec-928b-4b59f2013f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:23.499651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 921/1855] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:23.504104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6f1bb-e98f-4067-aaad-5e210529795e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.527027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6f1bb-e98f-4067-aaad-5e210529795e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:23.532541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:23.533213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.533509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.533784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.535855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:23.536377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:23.538567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:23.558518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 935/1856] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:23.559606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceea210-16d9-4eec-928b-4b59f2013f50 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.580676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceea210-16d9-4eec-928b-4b59f2013f50 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:23.586061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:23.587087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.587412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.587680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.588404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:23.590574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:23.627163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.627749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 922/1857] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:23 2022] POST /identity/v3/projects => generated 409 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:23.640392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b45e74-79d7-452c-965b-1ad931c8fb8d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:23.706831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:23.707650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.707999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.708344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.709127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:23.711245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:23.736250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 936/1858] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:23 2022] POST /identity/v3/users => generated 305 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:23.740443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.754754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8efc1-5345-4110-9ffb-aaf22fd3b988 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:23.760973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:23.761740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.762126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.762294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.763043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:23.764968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:23.779427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 923/1859] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:23.785194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.796193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350e23-623d-42ef-aabe-5be14122b08a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:23.800954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76049bc6-f4a5-47a0-9ea3-6fafc7b86e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:23.801646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049bc6-f4a5-47a0-9ea3-6fafc7b86e26 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.801933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049bc6-f4a5-47a0-9ea3-6fafc7b86e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.802212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049bc6-f4a5-47a0-9ea3-6fafc7b86e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.802967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76049bc6-f4a5-47a0-9ea3-6fafc7b86e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:23.804628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76049bc6-f4a5-47a0-9ea3-6fafc7b86e26 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:23.815897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 937/1860] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:23.820107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.830890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0332-adf9-497f-a5d4-2bc6880d77b3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:23.835464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:23.836409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.836693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.837019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] PATH_INFO: `/v3/projects/e67f3c0e86724a83a407eb643bb91491/users/f317800a4efe4a75a2c478bbb9344a1e/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.852839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67f3c0e86724a83a407eb643bb91491, user_id=f317800a4efe4a75a2c478bbb9344a1e, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:23.855666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:23.870946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 924/1861] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:23 2022] PUT /identity/v3/projects/e67f3c0e86724a83a407eb643bb91491/users/f317800a4efe4a75a2c478bbb9344a1e/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:23.882295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:23.882709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:23.883060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:23.914306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] MFA Rules not processed for user `f317800a4efe4a75a2c478bbb9344a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:23.915569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:23.954523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:23.984888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 938/1862] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:23.992153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.992305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:23.993710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.054773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb6571-2cdd-4801-b7b5-2a538c70748c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.059886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.060605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.060893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.061098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92741952-627e-4c6f-a39f-d191b2fe77c6 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.061197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.061863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.063821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.065816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.066699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.066983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.067253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] PATH_INFO: `/v3/users/ce9baecbe1ee469a9a424d4d93e9e53c` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.068486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9baecbe1ee469a9a424d4d93e9e53c)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.070403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.093370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 925/1863] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:23 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:24.094704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.115682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:24.122747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9baecbe1ee469a9a424d4d93e9e53c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.135717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9baecbe1ee469a9a424d4d93e9e53c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.144385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9baecbe1ee469a9a424d4d93e9e53c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.151493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9baecbe1ee469a9a424d4d93e9e53c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.152216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-adec1961-97aa-4b81-b1cf-92eb528b9a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9baecbe1ee469a9a424d4d93e9e53c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.158192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 939/1864] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:23 2022] DELETE /identity/v3/users/ce9baecbe1ee469a9a424d4d93e9e53c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:24.162359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18220874-d9e5-4ff6-8d10-8db79df994c7 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.162490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.162768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.163050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.179508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:24.180714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.218892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.219608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.219895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.220171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.220866 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.222818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.234520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:24.243577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 926/1865] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:24 2022] POST /identity/v3/users => generated 311 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:24.247101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 940/1866] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:24.248554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.252164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.256328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.257778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.264817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6fca-02e3-494b-8ea9-3333b38c2687 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.269721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.270416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.270686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.271003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.271778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.273391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.283510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 927/1867] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:24.287760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.297833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc30a8-155c-493c-ae46-bf90d91e609b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.302530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.303495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.303780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.304055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] PATH_INFO: `/v3/projects/04f804d8add94b159d02e800b5ecf273/users/1413fe586ad64e8d8b9fa784789df53e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.326936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f804d8add94b159d02e800b5ecf273, user_id=1413fe586ad64e8d8b9fa784789df53e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.330893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.343653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89446d4-ebf9-467c-917f-04d2ceb62a8e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.353937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 928/1868] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:24 2022] PUT /identity/v3/projects/04f804d8add94b159d02e800b5ecf273/users/1413fe586ad64e8d8b9fa784789df53e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:24.355111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.357155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.456411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.457192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.457503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.457784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.459117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb13447-55fc-470d-a818-d931c287186e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.471695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.472705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.473532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.473867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] PATH_INFO: `/v3/users/c1686304f4c14f17abc01b06cce6f864` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.475029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.478080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.479642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1686304f4c14f17abc01b06cce6f864)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.485023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.489683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 941/1869] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:24.490620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.502499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1571674-f048-43b6-9eba-c2d742f81c71 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.508254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.509172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.509284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.509482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.510222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.512734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.526903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 942/1870] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:24.529138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.540645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074528d3-476d-4c28-b9d0-3c4d79bb08e5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.548124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258e24-45d2-4186-98a8-526766584d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.549138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258e24-45d2-4186-98a8-526766584d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.549483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258e24-45d2-4186-98a8-526766584d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.549764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258e24-45d2-4186-98a8-526766584d3b admin admin] PATH_INFO: `/v3/projects/04f804d8add94b159d02e800b5ecf273/users/1413fe586ad64e8d8b9fa784789df53e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.551964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258e24-45d2-4186-98a8-526766584d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f804d8add94b159d02e800b5ecf273, user_id=1413fe586ad64e8d8b9fa784789df53e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:24.557205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258e24-45d2-4186-98a8-526766584d3b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:24.570505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:24.573890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 943/1871] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:24 2022] PUT /identity/v3/projects/04f804d8add94b159d02e800b5ecf273/users/1413fe586ad64e8d8b9fa784789df53e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:24.584205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.584627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.585075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.628897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1686304f4c14f17abc01b06cce6f864'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.651319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1686304f4c14f17abc01b06cce6f864'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.662896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] MFA Rules not processed for user `1413fe586ad64e8d8b9fa784789df53e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:24.665012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.676110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1686304f4c14f17abc01b06cce6f864'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.691847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1686304f4c14f17abc01b06cce6f864'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.693084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b0cafc21-b4bf-4eaa-a424-872d664ad533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1686304f4c14f17abc01b06cce6f864'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:24.703878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 929/1872] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:24 2022] DELETE /identity/v3/users/c1686304f4c14f17abc01b06cce6f864 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:24.709059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.709656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.710752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.738149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:24.740144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.846180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 930/1873] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:24.846544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 944/1874] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:24.850327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.852451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.891204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:24.893174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:24.922104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:24.928157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe9516-ff42-4f22-b16a-dabf152a7022 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.932651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:24.933703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:24.934043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:24.934393 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:24.977030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae985a9-95b1-4238-b3f5-28937b631b14 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:24.994773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:25.009428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:25.011023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:25.025069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 931/1875] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:24 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:25.126689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:25.126689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:25.126689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:25.126689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:25.171780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:25.176313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:25.209396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 945/1876] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:25.357915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:25.366804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2934d42-8e6d-45d6-a587-2a65af04fead service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:25.371310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:25.372079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:25.372423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:25.372706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:25.374942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:25.483737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:25.485248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:25.485707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:25.487360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:25.501998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 932/1877] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:25.509335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0ab79-541a-497f-acdc-ba4a73954306 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:25.516984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:25.517959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:25.518244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:25.518563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] PATH_INFO: `/v3/projects/c95ed362b3bc4725abea964176ff19b9` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:25.520020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95ed362b3bc4725abea964176ff19b9)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:25.522068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:25.585089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95ed362b3bc4725abea964176ff19b9', 'request_id': 'req-1d675ba1-9174-413e-84ef-80093a0f8817'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.591454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95ed362b3bc4725abea964176ff19b9', 'request_id': 'req-1d675ba1-9174-413e-84ef-80093a0f8817'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.605244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95ed362b3bc4725abea964176ff19b9', 'request_id': 'req-1d675ba1-9174-413e-84ef-80093a0f8817'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.634442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] The token cache is being invalidate because project c95ed362b3bc4725abea964176ff19b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:25.634738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.636389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.644080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 946/1878] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:25 2022] DELETE /identity/v3/projects/c95ed362b3bc4725abea964176ff19b9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:25.645166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:25.658939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71babb9c-8fa7-4481-ae6d-2e06d840846a service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:25.735950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:25.736874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:25.737235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:25.737493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] PATH_INFO: `/v3/users/2c2aeaa1cd9d48ea84a25465a7afba30` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:25.740299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2aeaa1cd9d48ea84a25465a7afba30)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:25.744214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:25.793968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:25.795712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:25.823978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d675ba1-9174-413e-84ef-80093a0f8817 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:25.832105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:25.833155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:25.833527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:25.833993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] PATH_INFO: `/v3/projects/01072eb5a4aa42b0842d3f4faecabe8f` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:25.835353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01072eb5a4aa42b0842d3f4faecabe8f)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:25.838060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:25.861162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2aeaa1cd9d48ea84a25465a7afba30'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.889226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2aeaa1cd9d48ea84a25465a7afba30'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.914152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2aeaa1cd9d48ea84a25465a7afba30'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.934634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2aeaa1cd9d48ea84a25465a7afba30'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.935359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2aeaa1cd9d48ea84a25465a7afba30'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.948278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 933/1879] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:25 2022] DELETE /identity/v3/users/2c2aeaa1cd9d48ea84a25465a7afba30 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:25.962449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01072eb5a4aa42b0842d3f4faecabe8f', 'request_id': 'req-25129c05-0f85-4f89-be43-a8d891b7b431'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.981390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01072eb5a4aa42b0842d3f4faecabe8f', 'request_id': 'req-25129c05-0f85-4f89-be43-a8d891b7b431'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:25.983006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:25.993762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01072eb5a4aa42b0842d3f4faecabe8f', 'request_id': 'req-25129c05-0f85-4f89-be43-a8d891b7b431'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.028620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] The token cache is being invalidate because project 01072eb5a4aa42b0842d3f4faecabe8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:26.031104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.033867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.035727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 947/1880] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:25 2022] DELETE /identity/v3/projects/01072eb5a4aa42b0842d3f4faecabe8f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:26.039669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:26.041757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:26.055581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:26.146493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc7f55-c509-4bd0-b5da-f8ce173838ac admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:26.154680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:26.156363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:26.156657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:26.156937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] PATH_INFO: `/v3/users/fd9e38583e8647ad9b1ae8dd7791e1bf` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:26.158199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9e38583e8647ad9b1ae8dd7791e1bf)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:26.161391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:26.230458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9e38583e8647ad9b1ae8dd7791e1bf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.250058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9e38583e8647ad9b1ae8dd7791e1bf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.259536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9e38583e8647ad9b1ae8dd7791e1bf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.270892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9e38583e8647ad9b1ae8dd7791e1bf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.274467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-599bc15a-7332-4e5c-95c3-3cabae166e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9e38583e8647ad9b1ae8dd7791e1bf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.285104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 934/1881] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:26 2022] DELETE /identity/v3/users/fd9e38583e8647ad9b1ae8dd7791e1bf => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:26.475321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:26.476996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:26.706325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25129c05-0f85-4f89-be43-a8d891b7b431 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:26.773211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:26.774424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:26.774808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:26.775088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] PATH_INFO: `/v3/projects/301bb81b756f46a0b1bdfe93bdac44c6` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:26.776652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=301bb81b756f46a0b1bdfe93bdac44c6)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:26.778967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:26.829514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301bb81b756f46a0b1bdfe93bdac44c6', 'request_id': 'req-53bb2487-3158-4297-b4eb-62cf73b15fd0'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.835849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301bb81b756f46a0b1bdfe93bdac44c6', 'request_id': 'req-53bb2487-3158-4297-b4eb-62cf73b15fd0'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.842905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301bb81b756f46a0b1bdfe93bdac44c6', 'request_id': 'req-53bb2487-3158-4297-b4eb-62cf73b15fd0'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.863924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] The token cache is being invalidate because project 301bb81b756f46a0b1bdfe93bdac44c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:26.864303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.865087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:26.867048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 948/1882] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:26 2022] DELETE /identity/v3/projects/301bb81b756f46a0b1bdfe93bdac44c6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:26.873390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f2cac-2ac7-4735-85ae-e4c9bf589994 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:26.873857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f2cac-2ac7-4735-85ae-e4c9bf589994 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:26.873857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f2cac-2ac7-4735-85ae-e4c9bf589994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:26.876705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:26.878151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:26.895514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-705f2cac-2ac7-4735-85ae-e4c9bf589994 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:26.896804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-705f2cac-2ac7-4735-85ae-e4c9bf589994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:26.911075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:26.957304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 935/1883] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:26.957740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb2487-3158-4297-b4eb-62cf73b15fd0 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:26.962470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:26.963426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:26.963709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:26.963980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] PATH_INFO: `/v3/projects/b42ae2e961344774b3e1ef9c74fd1e16` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:26.965789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42ae2e961344774b3e1ef9c74fd1e16)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:26.965971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04272-5221-4a0a-83d4-164ffe4777b6 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:26.966263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04272-5221-4a0a-83d4-164ffe4777b6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:26.966543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04272-5221-4a0a-83d4-164ffe4777b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:26.968131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:26.984806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-53b04272-5221-4a0a-83d4-164ffe4777b6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:26.985932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-53b04272-5221-4a0a-83d4-164ffe4777b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:26.993810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 936/1884] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:26.998111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04272-5221-4a0a-83d4-164ffe4777b6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.007696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04272-5221-4a0a-83d4-164ffe4777b6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.022471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42ae2e961344774b3e1ef9c74fd1e16', 'request_id': 'req-5293492f-7ef2-42b7-975f-427cf6a4c98b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:27.024060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:27.030308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42ae2e961344774b3e1ef9c74fd1e16', 'request_id': 'req-5293492f-7ef2-42b7-975f-427cf6a4c98b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:27.036515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42ae2e961344774b3e1ef9c74fd1e16', 'request_id': 'req-5293492f-7ef2-42b7-975f-427cf6a4c98b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:27.054492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] The token cache is being invalidate because project b42ae2e961344774b3e1ef9c74fd1e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:27.054799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:27.055552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-5293492f-7ef2-42b7-975f-427cf6a4c98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:27.057576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 949/1885] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:26 2022] DELETE /identity/v3/projects/b42ae2e961344774b3e1ef9c74fd1e16 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:27.062779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15420a-d815-4c21-ba1a-ea40f4174ee4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.063087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15420a-d815-4c21-ba1a-ea40f4174ee4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.063389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15420a-d815-4c21-ba1a-ea40f4174ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.076023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.077014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.077525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.078005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.078948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:27.079620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.081583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.082081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-cc15420a-d815-4c21-ba1a-ea40f4174ee4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:27.083299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-cc15420a-d815-4c21-ba1a-ea40f4174ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:27.091642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 950/1886] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:27.100183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 937/1887] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:52:27.100315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af910834-02b7-48df-bf91-614709064980 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.100315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af910834-02b7-48df-bf91-614709064980 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.100315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af910834-02b7-48df-bf91-614709064980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.101139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.103119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:27.121155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1d5ac-15ea-442e-a423-4818b0ff6923 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.126505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.128306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.128579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.128868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.129705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.132066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.134378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-af910834-02b7-48df-bf91-614709064980 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:27.136534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-af910834-02b7-48df-bf91-614709064980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:27.145821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 951/1888] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:27.154265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af910834-02b7-48df-bf91-614709064980 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.171413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 938/1889] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:27.171648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af910834-02b7-48df-bf91-614709064980 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.173799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.253404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fdea9-1981-438d-84fa-71ac23295982 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.260569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.261117 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.261475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.261783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.262089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.262190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.262373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.262699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.263055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.263476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:27.263835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.265284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.265458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.280403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 952/1890] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:52:27.284320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.299656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbce287-fc9d-419c-99d4-18c663811ade admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.306005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 939/1891] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:27.307327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.309048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.310432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.310762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.311074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.311878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.313876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.320424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c58514-34fa-43c2-9b94-47a00b09b7cc admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.326142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.327528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.327949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.328232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.328986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.331401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.345042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 940/1892] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:27.349099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.355341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 953/1893] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:27.356101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.360184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a21db-eca5-402b-bd05-027691c63ee6 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.372142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:27.431636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.434671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.438356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.438898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.440595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.444189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.465602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 941/1894] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:27.469773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.484288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fab0b-9b68-49b8-b9ab-dfb50a356764 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.487527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed97d79-b0f0-4b07-86db-c326d5319c99 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.491252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856e816-0361-4b33-8537-d836872f8eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.492208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856e816-0361-4b33-8537-d836872f8eac admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.493003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856e816-0361-4b33-8537-d836872f8eac admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.493344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856e816-0361-4b33-8537-d836872f8eac admin admin] PATH_INFO: `/v3/projects/ae6ff0d6bca84ad699dfcace2f456a7b/users/d5d966e563e14c5292cdb683b75eb7d5/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.496439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:27.498716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.499460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.502094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.502094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.502094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.502999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.516777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856e816-0361-4b33-8537-d836872f8eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6ff0d6bca84ad699dfcace2f456a7b, user_id=d5d966e563e14c5292cdb683b75eb7d5, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.520132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856e816-0361-4b33-8537-d836872f8eac admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.551862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 954/1895] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:27.555412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.555535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 942/1896] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:27 2022] PUT /identity/v3/projects/ae6ff0d6bca84ad699dfcace2f456a7b/users/d5d966e563e14c5292cdb683b75eb7d5/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:27.569257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc6827-e08d-431e-b334-50fed0baf6ea None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.569578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc6827-e08d-431e-b334-50fed0baf6ea None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.569881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc6827-e08d-431e-b334-50fed0baf6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.594454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c9c31-a794-4238-a919-58030a8fcc78 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.604287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c5bc6827-e08d-431e-b334-50fed0baf6ea None None] MFA Rules not processed for user `d5d966e563e14c5292cdb683b75eb7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:27.606052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c5bc6827-e08d-431e-b334-50fed0baf6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:27.698939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.700076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.700562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.701094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.701848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.703518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.711324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 943/1897] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:27.715836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc6827-e08d-431e-b334-50fed0baf6ea None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.719420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 955/1898] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:27.724424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.727006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc6827-e08d-431e-b334-50fed0baf6ea None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.734202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.734202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.734202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.734202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.734786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.736640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.750595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a678b0-0d69-41ea-880f-3347960eba6c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.758525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.761892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.762575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.763041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.764906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.767298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.768822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 944/1899] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:27.772794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.784473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 956/1900] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:27.788311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.799681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663d5ef-1acb-46b0-a612-be1393f7d267 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.870289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ddd8f-f32b-4aab-87d6-e3901fd1e3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.871464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ddd8f-f32b-4aab-87d6-e3901fd1e3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.871672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ddd8f-f32b-4aab-87d6-e3901fd1e3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.871946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ddd8f-f32b-4aab-87d6-e3901fd1e3d8 admin admin] PATH_INFO: `/v3/projects/77bdb11ccc2348f88f11e57660c66d2c/users/d4a1bdf7e2814158b2e3e3a2d86a2627/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.883655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f29eeb-9d90-4bd3-83cb-a162a6058622 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.889993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:27.890801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.890911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ddd8f-f32b-4aab-87d6-e3901fd1e3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bdb11ccc2348f88f11e57660c66d2c, user_id=d4a1bdf7e2814158b2e3e3a2d86a2627, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.891095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.891468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.892186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:27.893780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ddd8f-f32b-4aab-87d6-e3901fd1e3d8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.894319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:27.915833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 957/1901] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:27 2022] PUT /identity/v3/projects/77bdb11ccc2348f88f11e57660c66d2c/users/d4a1bdf7e2814158b2e3e3a2d86a2627/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:27.927471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322fc2a-7b5d-4cbf-b7c2-300b024a89b8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:27.927956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322fc2a-7b5d-4cbf-b7c2-300b024a89b8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:27.928343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322fc2a-7b5d-4cbf-b7c2-300b024a89b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:27.936055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 945/1902] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:27.940762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:27.948422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:27.959271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c66104-1323-48f6-8918-b95f9efda0c6 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:27.969066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b322fc2a-7b5d-4cbf-b7c2-300b024a89b8 None None] MFA Rules not processed for user `d4a1bdf7e2814158b2e3e3a2d86a2627`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:27.970721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b322fc2a-7b5d-4cbf-b7c2-300b024a89b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:28.047201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:28.073363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.074307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.074593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.074925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.075705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.077543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.087267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 958/1903] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:28.090031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322fc2a-7b5d-4cbf-b7c2-300b024a89b8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.093506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 946/1904] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:28.098229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.104275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322fc2a-7b5d-4cbf-b7c2-300b024a89b8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.113227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.113672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6f73-8f7c-4aba-ac1a-19f5b90b6c9d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.114912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.115478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.116001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.116899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.119603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.120298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.120691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.121121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.121724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.122703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.124911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.142592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 947/1905] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:28.146279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.158538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd6e13-2180-4027-9f35-514ef346ae64 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.163963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2363fce-f3d9-4022-b4b5-0906c4274d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.164288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 959/1906] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:28 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:28.165000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363fce-f3d9-4022-b4b5-0906c4274d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.165444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363fce-f3d9-4022-b4b5-0906c4274d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.165783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363fce-f3d9-4022-b4b5-0906c4274d7d admin admin] PATH_INFO: `/v3/projects/096b2d0441cb4a48a6ef5d7ec293b13d/users/059d021bdaae452bbe07ceec663b1b6f/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.169198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.182919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2363fce-f3d9-4022-b4b5-0906c4274d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=096b2d0441cb4a48a6ef5d7ec293b13d, user_id=059d021bdaae452bbe07ceec663b1b6f, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.185817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2363fce-f3d9-4022-b4b5-0906c4274d7d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.202153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 948/1907] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:28 2022] PUT /identity/v3/projects/096b2d0441cb4a48a6ef5d7ec293b13d/users/059d021bdaae452bbe07ceec663b1b6f/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:28.211993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8ec10-f6cf-401f-b4de-9c95e6b952d3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.212288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8ec10-f6cf-401f-b4de-9c95e6b952d3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.212564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8ec10-f6cf-401f-b4de-9c95e6b952d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.240553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-7ef8ec10-f6cf-401f-b4de-9c95e6b952d3 None None] MFA Rules not processed for user `059d021bdaae452bbe07ceec663b1b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:28.241789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7ef8ec10-f6cf-401f-b4de-9c95e6b952d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:28.242365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a180e39-cc29-4447-88d7-cf84aa3ba208 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.307448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.308728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.309857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.310206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.311007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.313000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.316907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 949/1908] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:28.317716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8ec10-f6cf-401f-b4de-9c95e6b952d3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.329399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8ec10-f6cf-401f-b4de-9c95e6b952d3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.337009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.337009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.337009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.337009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.337580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.337784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 960/1909] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:28 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:28.342977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.343843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.356397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61e0c-899e-4cf0-9fa8-5942a77fdc34 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.361981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.362670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.362954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.363232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.364010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.365652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.372009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:28.382818 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 961/1910] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:28.383721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.391531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 950/1911] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:28 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:28.394262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.397773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2421e6b-7c2f-44da-9a0c-506ee3be1f47 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.460900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.461802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.462079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.462350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] PATH_INFO: `/v3/projects/d72ffb7132c94c1990bd19481a782057/users/cf86518de9fa4280b2b3a8e53457c2e5/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.465489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c544305-fa35-47af-a7f5-3fa7d7ca9d04 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.470972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.471711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.472000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.472279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.472983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.474928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.478072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72ffb7132c94c1990bd19481a782057, user_id=cf86518de9fa4280b2b3a8e53457c2e5, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.480986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.492130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:28.497380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 962/1912] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:28 2022] PUT /identity/v3/projects/d72ffb7132c94c1990bd19481a782057/users/cf86518de9fa4280b2b3a8e53457c2e5/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:28.501441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.502211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 951/1913] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:28 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:28.503097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.513287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8476256-daf9-432e-9d8e-a4c2ca754094 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.565485 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aef806-3acb-44fe-b358-4988294700ed admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.569237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.570051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.570238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.570346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.570605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.570911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.571192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.571353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.571503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.572242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.573600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.573841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.587461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 963/1915] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:28.587633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 952/1915] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:28.588762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.588901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.599154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074394ac-f44b-4935-bf69-6fc966dae702 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.599409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb77efb-e43f-4215-a091-b7d99b2722ac admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.603753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7faa2-ec7d-4c96-8679-dd25ca8f3e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.604041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.604627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7faa2-ec7d-4c96-8679-dd25ca8f3e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.604908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7faa2-ec7d-4c96-8679-dd25ca8f3e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.605047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.605210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7faa2-ec7d-4c96-8679-dd25ca8f3e44 admin admin] PATH_INFO: `/v3/projects/d72ffb7132c94c1990bd19481a782057/users/cf86518de9fa4280b2b3a8e53457c2e5/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.605364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.605552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] PATH_INFO: `/v3/projects/73ab39624af742c3b379a818865c5620/users/ebe15545e66c4a9e95fb3333e5f7b064/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.607290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7faa2-ec7d-4c96-8679-dd25ca8f3e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72ffb7132c94c1990bd19481a782057, user_id=cf86518de9fa4280b2b3a8e53457c2e5, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.610047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7faa2-ec7d-4c96-8679-dd25ca8f3e44 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.620956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ab39624af742c3b379a818865c5620, user_id=ebe15545e66c4a9e95fb3333e5f7b064, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.621288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 953/1916] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:28 2022] PUT /identity/v3/projects/d72ffb7132c94c1990bd19481a782057/users/cf86518de9fa4280b2b3a8e53457c2e5/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:28.623796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.632840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.633145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.633430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.638087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 964/1917] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:28 2022] PUT /identity/v3/projects/73ab39624af742c3b379a818865c5620/users/ebe15545e66c4a9e95fb3333e5f7b064/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:28.641993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.664396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] MFA Rules not processed for user `cf86518de9fa4280b2b3a8e53457c2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:28.665634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:28.715978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:28.723237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b57ec-76f4-4d20-af65-d69d4c9f3fb5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.732602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.734044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.734329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.734603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.735711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.737370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.752013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 965/1918] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:28.756081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.767741 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc7955-61c3-4783-8266-58334af901f5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.772744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5eff15-5aa2-414f-bc1c-8d3833b12141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.773944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eff15-5aa2-414f-bc1c-8d3833b12141 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.773944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eff15-5aa2-414f-bc1c-8d3833b12141 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.774209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eff15-5aa2-414f-bc1c-8d3833b12141 admin admin] PATH_INFO: `/v3/projects/73ab39624af742c3b379a818865c5620/users/ebe15545e66c4a9e95fb3333e5f7b064/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.779989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 954/1919] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:28.788353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5eff15-5aa2-414f-bc1c-8d3833b12141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ab39624af742c3b379a818865c5620, user_id=ebe15545e66c4a9e95fb3333e5f7b064, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.789043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.790580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5eff15-5aa2-414f-bc1c-8d3833b12141 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.792248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:28.814306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 966/1920] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:28 2022] PUT /identity/v3/projects/73ab39624af742c3b379a818865c5620/users/ebe15545e66c4a9e95fb3333e5f7b064/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:28.820065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.821027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.821321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.852998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a322d4-824c-4aa7-9910-a0576b58e338 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.854635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] MFA Rules not processed for user `ebe15545e66c4a9e95fb3333e5f7b064`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:28.855958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:28.878321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:28.905823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.906586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.906920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.907221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.938077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 967/1921] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:28.950129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:28.951687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:28.963650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd11280-3d5e-4dbd-92c1-435144b88eb0 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:28.967660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:28.968303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:28.968440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:28.968586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:28.968866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:28.969897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:28.979630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 955/1922] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:29.031003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:29.032484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:29.042177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 968/1923] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:29.335962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:29.359541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef8b7c-bb37-4598-976a-7a0107baeae9 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:29.364856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf86518de9fa4280b2b3a8e53457c2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd72ffb7132c94c1990bd19481a782057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:29.365550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:29.365850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:29.366137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:29.367030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:29.367348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:29.369004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:29.386778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 956/1924] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:29.387991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:29.407651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3217d7d-2075-4723-ac5f-1c34b5244b29 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:29.409677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf86518de9fa4280b2b3a8e53457c2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd72ffb7132c94c1990bd19481a782057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:29.410420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:29.410772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:29.411166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:29.411996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:29.414365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:29.442386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 969/1925] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:29 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:29.446544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:29.458381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93bbd0-1550-4e4f-b28e-21ed6db4cfcb tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:29.520489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf86518de9fa4280b2b3a8e53457c2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd72ffb7132c94c1990bd19481a782057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:29.521176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:29.521466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:29.521746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:29.522519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:29.522832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:29.524542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:29.536252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 957/1926] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:29.572570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:29.583295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57db26-be43-4412-b180-ef106f7bfa9d tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:29.587914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf86518de9fa4280b2b3a8e53457c2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd72ffb7132c94c1990bd19481a782057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:29.588804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:29.589077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:29.589355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] PATH_INFO: `/v3/projects/419c29d091d8487bbf3ab85f494d73ca` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:29.590647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorizing `identity:delete_project(project_id=419c29d091d8487bbf3ab85f494d73ca)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:29.593250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:29.630794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:29.642422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419c29d091d8487bbf3ab85f494d73ca', 'request_id': 'req-ac1c2580-7243-40c8-ba08-1171dbb43f6e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:29.649424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419c29d091d8487bbf3ab85f494d73ca', 'request_id': 'req-ac1c2580-7243-40c8-ba08-1171dbb43f6e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:29.654919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419c29d091d8487bbf3ab85f494d73ca', 'request_id': 'req-ac1c2580-7243-40c8-ba08-1171dbb43f6e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:29.668532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] The token cache is being invalidate because project 419c29d091d8487bbf3ab85f494d73ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:29.668822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:29.669563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ac1c2580-7243-40c8-ba08-1171dbb43f6e tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:29.671320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 970/1927] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:29 2022] DELETE /identity/v3/projects/419c29d091d8487bbf3ab85f494d73ca => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:29.871237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:29.873463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:29.953310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:29.966092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fc8e5-1f1f-421e-acc2-7a6dfe8fa38b tempest-VolumeQuotaClassesTest-2030296580 tempest-VolumeQuotaClassesTest-2030296580-project-admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:30.037114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:30.038060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:30.038341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:30.038617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] PATH_INFO: `/v3/users/d4a1bdf7e2814158b2e3e3a2d86a2627` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:30.039851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a1bdf7e2814158b2e3e3a2d86a2627)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:30.041828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:30.091300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a1bdf7e2814158b2e3e3a2d86a2627'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:30.104637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a1bdf7e2814158b2e3e3a2d86a2627'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:30.118756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a1bdf7e2814158b2e3e3a2d86a2627'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:30.130237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a1bdf7e2814158b2e3e3a2d86a2627'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:30.131034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a1bdf7e2814158b2e3e3a2d86a2627'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:30.137446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 958/1928] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:29 2022] DELETE /identity/v3/users/d4a1bdf7e2814158b2e3e3a2d86a2627 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:30.142049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:30.142639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:30.142639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:30.160318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:30.161646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:30.240468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 971/1929] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:30.251323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:30.253557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:30.341870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205a3cb-6fb2-4fdc-91fa-471745d5797e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:30.346641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:30.347315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:30.347628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:30.347913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:30.360238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:30.362022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:30.374200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 959/1930] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:30.758063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:30.759538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:30.900230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394343e-273d-4ab9-8b9b-e894b501e068 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:30.909862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:30.910516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:30.912655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:30.912982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:30.927150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:30.956706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:30.958825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:30.986388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3035027-e0c3-4663-85c2-a19145884d83 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:31.000128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:31.000468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:31.000853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:31.001133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] PATH_INFO: `/v3/projects/77bdb11ccc2348f88f11e57660c66d2c` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:31.005754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77bdb11ccc2348f88f11e57660c66d2c)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:31.011140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:31.056952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:31.058953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:31.071347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 972/1931] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:31.126333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:31.222649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77bdb11ccc2348f88f11e57660c66d2c', 'request_id': 'req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.231544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77bdb11ccc2348f88f11e57660c66d2c', 'request_id': 'req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.238793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77bdb11ccc2348f88f11e57660c66d2c', 'request_id': 'req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.259503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] The token cache is being invalidate because project 77bdb11ccc2348f88f11e57660c66d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:31.260039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.261024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.264801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 960/1932] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:30 2022] DELETE /identity/v3/projects/77bdb11ccc2348f88f11e57660c66d2c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:31.268605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:31.282788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e3792-fed7-4844-a88d-6bc0bf2ddf8b service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:31.303934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:31.368965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:31.369863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:31.370142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:31.370426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] PATH_INFO: `/v3/users/cf86518de9fa4280b2b3a8e53457c2e5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:31.371865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf86518de9fa4280b2b3a8e53457c2e5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:31.373975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:31.429007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf86518de9fa4280b2b3a8e53457c2e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.442359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf86518de9fa4280b2b3a8e53457c2e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.450358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf86518de9fa4280b2b3a8e53457c2e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.459619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf86518de9fa4280b2b3a8e53457c2e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.460334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf86518de9fa4280b2b3a8e53457c2e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:31.471420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 973/1933] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:31 2022] DELETE /identity/v3/users/cf86518de9fa4280b2b3a8e53457c2e5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:31.817613 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:31.819271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:31.904414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9dcfe-de4e-4a34-9069-5e79d6edc7e0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:31.973125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75016ea1-f5a5-4230-b5a8-aeb0ab324106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:31.974086 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75016ea1-f5a5-4230-b5a8-aeb0ab324106 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:31.974768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75016ea1-f5a5-4230-b5a8-aeb0ab324106 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:31.975152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75016ea1-f5a5-4230-b5a8-aeb0ab324106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:31.977526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-75016ea1-f5a5-4230-b5a8-aeb0ab324106 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:32.046461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75016ea1-f5a5-4230-b5a8-aeb0ab324106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:32.048101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75016ea1-f5a5-4230-b5a8-aeb0ab324106 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:32.058083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 961/1934] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:32.285589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:32.287296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:32.376297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b66d6c-b6a0-4213-bd73-ada26d1d4589 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:32.382141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:32.383106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:32.386550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:32.386884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] PATH_INFO: `/v3/projects/d72ffb7132c94c1990bd19481a782057` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:32.388462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72ffb7132c94c1990bd19481a782057)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:32.391337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:32.449360 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:32.459528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72ffb7132c94c1990bd19481a782057', 'request_id': 'req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:32.471092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72ffb7132c94c1990bd19481a782057', 'request_id': 'req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:32.478628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72ffb7132c94c1990bd19481a782057', 'request_id': 'req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:32.495538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] The token cache is being invalidate because project d72ffb7132c94c1990bd19481a782057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:32.495841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:32.496767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fdb65af-7bab-46ca-9ea6-cb81c58a0965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:32.498802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 974/1935] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:32 2022] DELETE /identity/v3/projects/d72ffb7132c94c1990bd19481a782057 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:32.503865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98828915-b7bf-41e4-a0c5-8a897a47400d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:32.504367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98828915-b7bf-41e4-a0c5-8a897a47400d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:32.504865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98828915-b7bf-41e4-a0c5-8a897a47400d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:32.525410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-98828915-b7bf-41e4-a0c5-8a897a47400d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:32.526765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-98828915-b7bf-41e4-a0c5-8a897a47400d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:32.533119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:32.601150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 962/1936] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:32.609126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80f2c0-0984-4b03-b527-73ce702559c7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:32.609443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80f2c0-0984-4b03-b527-73ce702559c7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:32.609732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80f2c0-0984-4b03-b527-73ce702559c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:32.630462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-fc80f2c0-0984-4b03-b527-73ce702559c7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:32.631634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fc80f2c0-0984-4b03-b527-73ce702559c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:32.642286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 975/1937] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:32.646815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98828915-b7bf-41e4-a0c5-8a897a47400d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:32.657172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98828915-b7bf-41e4-a0c5-8a897a47400d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:32.661814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:32.662521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:32.662804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:32.663212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:32.663991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:32.664302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:32.666071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:32.681910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 963/1938] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:32.683803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80f2c0-0984-4b03-b527-73ce702559c7 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:32.695731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80f2c0-0984-4b03-b527-73ce702559c7 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:32.701193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:32.701933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:32.702219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:32.702508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:32.703211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:32.705576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:32.759459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 976/1939] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:32 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:32.761237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:32.775741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f7888-63da-4bf9-a895-4fdef4970730 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:32.879739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:32.880767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:32.880981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:32.881263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:32.882019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:32.884222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:32.938610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 964/1940] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:32 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:32.940190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:32.956498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebeae3-b6cd-44ed-8bde-468714f1288b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:32.973172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:32.974422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:32.974854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:32.975199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:32.976066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:32.977807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:32.995455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 977/1941] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:32.996144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.008497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ccc41-1450-49d4-a9f5-89f841f34628 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.014410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122e91c-e360-47d2-8ee1-45dc63e82b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.015452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122e91c-e360-47d2-8ee1-45dc63e82b51 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.015816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122e91c-e360-47d2-8ee1-45dc63e82b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.016104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122e91c-e360-47d2-8ee1-45dc63e82b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.016857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122e91c-e360-47d2-8ee1-45dc63e82b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.018558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122e91c-e360-47d2-8ee1-45dc63e82b51 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.030977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 965/1942] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:33.036235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.046877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59278943-45da-43c7-b473-1ea92f5b2494 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.052173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.053157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.053495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.053775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] PATH_INFO: `/v3/projects/12bc0ca514a24ea88dd5c55fc42c692a/users/7e62c03308a84bec87f2a612ae2004bd/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.075105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=12bc0ca514a24ea88dd5c55fc42c692a, user_id=7e62c03308a84bec87f2a612ae2004bd, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.083846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.118788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 978/1943] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:33 2022] PUT /identity/v3/projects/12bc0ca514a24ea88dd5c55fc42c692a/users/7e62c03308a84bec87f2a612ae2004bd/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:33.130357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb404d-ed4d-43ad-aacf-97a758f293b5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.130975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb404d-ed4d-43ad-aacf-97a758f293b5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.131476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb404d-ed4d-43ad-aacf-97a758f293b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.170740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-89eb404d-ed4d-43ad-aacf-97a758f293b5 None None] MFA Rules not processed for user `7e62c03308a84bec87f2a612ae2004bd`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:33.172383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-89eb404d-ed4d-43ad-aacf-97a758f293b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:33.258759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 966/1944] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:33.263939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.334808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cddd0-f123-4364-944d-f0ff1201e5bf admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.341050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.341786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.341992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.342262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.343332 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.345266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.373791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 979/1945] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:33 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:33.378075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb404d-ed4d-43ad-aacf-97a758f293b5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.389587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb404d-ed4d-43ad-aacf-97a758f293b5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.393875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:33.467873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.468650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.469067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.469492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.470813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.473115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.502360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 967/1946] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:33 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:33.506644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.520214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13fdcc-b904-4f38-96a7-f1abbcf96e65 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.528945 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.529614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.529858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.530144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.530890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.532605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.548326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 980/1947] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:33.553455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.578926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8d99-1609-4eff-81dc-2af7364fdf2e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.588520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.590546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.591427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.592155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] PATH_INFO: `/v3/projects/77105406561d4deeaad4b8d05a327d1f/users/6b934b48eb3f4acbb1b674e103f24588/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.618021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77105406561d4deeaad4b8d05a327d1f, user_id=6b934b48eb3f4acbb1b674e103f24588, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.621399 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.664627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 968/1948] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:33 2022] PUT /identity/v3/projects/77105406561d4deeaad4b8d05a327d1f/users/6b934b48eb3f4acbb1b674e103f24588/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:33.664800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.681050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a2715-aa42-492c-bf41-5bc9b0fac208 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.748507 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:33.781365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.782306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.782602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.782866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.783707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.785507 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.797979 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 981/1949] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:33.802799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.815250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d2fc7-2ce3-413e-b8f3-8a73bd79273b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.821021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5f5da-6b2c-4b49-90d1-588603ce2093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.827306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f5da-6b2c-4b49-90d1-588603ce2093 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.827595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.827702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f5da-6b2c-4b49-90d1-588603ce2093 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.828877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f5da-6b2c-4b49-90d1-588603ce2093 admin admin] PATH_INFO: `/v3/projects/77105406561d4deeaad4b8d05a327d1f/users/6b934b48eb3f4acbb1b674e103f24588/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.830495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:33.830608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5f5da-6b2c-4b49-90d1-588603ce2093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77105406561d4deeaad4b8d05a327d1f, user_id=6b934b48eb3f4acbb1b674e103f24588, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.834052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5f5da-6b2c-4b49-90d1-588603ce2093 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.846465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 969/1950] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:33 2022] PUT /identity/v3/projects/77105406561d4deeaad4b8d05a327d1f/users/6b934b48eb3f4acbb1b674e103f24588/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:33.850380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070139f5-c03d-4e38-a711-5b3aa1b368da admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:33.859082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.859660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.860143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.891101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] MFA Rules not processed for user `6b934b48eb3f4acbb1b674e103f24588`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:33.892335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:33.912540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:33.913720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:33.913997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:33.914270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] PATH_INFO: `/v3/users/f317800a4efe4a75a2c478bbb9344a1e` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:33.915542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f317800a4efe4a75a2c478bbb9344a1e)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:33.917398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:33.964539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 970/1951] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:33.967292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f317800a4efe4a75a2c478bbb9344a1e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:33.976014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:33.977449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:33.980282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f317800a4efe4a75a2c478bbb9344a1e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:33.989431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f317800a4efe4a75a2c478bbb9344a1e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:33.997550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f317800a4efe4a75a2c478bbb9344a1e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:33.998290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e682c0b3-f04b-45b0-aca4-a91b9e6b01a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f317800a4efe4a75a2c478bbb9344a1e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:34.004292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 982/1952] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:33 2022] DELETE /identity/v3/users/f317800a4efe4a75a2c478bbb9344a1e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:34.008882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:34.008996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:34.009257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:34.025199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:34.026954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:34.028953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:34.038667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f89609-24bb-461c-9fcd-0478aff27f37 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:34.081341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:34.096434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:34.097111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:34.097400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:34.097679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:34.099865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:34.101121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 983/1953] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:34.110504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:34.113028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:34.212747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:34.214281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:34.229896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb459-5dbc-4668-ae49-f38b0a45c724 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:34.235993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31cb7d-6325-4d74-8524-17e2c6f665db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:34.236854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31cb7d-6325-4d74-8524-17e2c6f665db service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:34.237307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31cb7d-6325-4d74-8524-17e2c6f665db service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:34.237719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31cb7d-6325-4d74-8524-17e2c6f665db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:34.239425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 971/1954] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:34.282490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31cb7d-6325-4d74-8524-17e2c6f665db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:34.284594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31cb7d-6325-4d74-8524-17e2c6f665db service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:34.301651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 984/1955] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:34.412656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:34.414107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:34.427757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb21a7-a043-4fcf-8446-b4bfd148f47f service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:34.432513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:34.433406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:34.433986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:34.434325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] PATH_INFO: `/v3/users/7e62c03308a84bec87f2a612ae2004bd` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:34.435582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e62c03308a84bec87f2a612ae2004bd)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:34.437562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:34.498611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e62c03308a84bec87f2a612ae2004bd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:34.524325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e62c03308a84bec87f2a612ae2004bd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:34.536566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e62c03308a84bec87f2a612ae2004bd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:34.546912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e62c03308a84bec87f2a612ae2004bd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:34.547768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e62c03308a84bec87f2a612ae2004bd'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:34.555750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 972/1956] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:34 2022] DELETE /identity/v3/users/7e62c03308a84bec87f2a612ae2004bd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:34.572096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:34.572626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:34.572898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:34.591082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:34.592815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:34.705756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 985/1957] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:34.725990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:34.728974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:34.782363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:34.829757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0b0f-00e2-4608-afa9-286e7a40cc6a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:34.836536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:34.837869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:34.840017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:34.840412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:34.852717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:34.854386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:34.870651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 973/1958] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:35.528372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:35.531095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:35.544376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9567433-bdf4-4390-ab07-55b24b56f6a8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:35.549858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:35.550795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:35.551084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:35.551468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] PATH_INFO: `/v3/projects/e67f3c0e86724a83a407eb643bb91491` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:35.552883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67f3c0e86724a83a407eb643bb91491)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:35.555200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:35.604564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:35.616199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:35.642295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401b09a-c0ab-4ffc-9a84-af2d75f78c9c service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:35.655681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67f3c0e86724a83a407eb643bb91491', 'request_id': 'req-7efea2c0-c08b-417f-b274-5676f36c90df'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.667564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67f3c0e86724a83a407eb643bb91491', 'request_id': 'req-7efea2c0-c08b-417f-b274-5676f36c90df'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.680210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67f3c0e86724a83a407eb643bb91491', 'request_id': 'req-7efea2c0-c08b-417f-b274-5676f36c90df'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.707699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] The token cache is being invalidate because project e67f3c0e86724a83a407eb643bb91491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:35.708244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.710738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.720595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 986/1959] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:35 2022] DELETE /identity/v3/projects/e67f3c0e86724a83a407eb643bb91491 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:35.722063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:35.722970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:35.723583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:35.723691 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] PATH_INFO: `/v3/projects/12bc0ca514a24ea88dd5c55fc42c692a` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:35.724642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:35.724952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12bc0ca514a24ea88dd5c55fc42c692a)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:35.726034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:35.726948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:35.740371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efea2c0-c08b-417f-b274-5676f36c90df admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:35.745426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:35.746457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:35.746877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:35.747159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] PATH_INFO: `/v3/users/1413fe586ad64e8d8b9fa784789df53e` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:35.748439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1413fe586ad64e8d8b9fa784789df53e)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:35.750372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:35.774374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:35.800730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12bc0ca514a24ea88dd5c55fc42c692a', 'request_id': 'req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.808216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12bc0ca514a24ea88dd5c55fc42c692a', 'request_id': 'req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.814901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1413fe586ad64e8d8b9fa784789df53e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.815513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12bc0ca514a24ea88dd5c55fc42c692a', 'request_id': 'req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.830353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1413fe586ad64e8d8b9fa784789df53e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.832178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] The token cache is being invalidate because project 12bc0ca514a24ea88dd5c55fc42c692a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:35.832469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.833095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.835314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 974/1960] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:35 2022] DELETE /identity/v3/projects/12bc0ca514a24ea88dd5c55fc42c692a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:35.839620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:35.841831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1413fe586ad64e8d8b9fa784789df53e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.841958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:35.850083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1413fe586ad64e8d8b9fa784789df53e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.850933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1413fe586ad64e8d8b9fa784789df53e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:35.857768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 987/1961] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:35 2022] DELETE /identity/v3/users/1413fe586ad64e8d8b9fa784789df53e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:35.913114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:35.922449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73fbc1-c6b7-4c24-b203-e86e5fc6671a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:35.991611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:35.992656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:35.993009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:35.993353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] PATH_INFO: `/v3/users/6b934b48eb3f4acbb1b674e103f24588` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:35.994652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b934b48eb3f4acbb1b674e103f24588)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:35.997288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:36.059486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b934b48eb3f4acbb1b674e103f24588'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.074901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b934b48eb3f4acbb1b674e103f24588'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.083709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b934b48eb3f4acbb1b674e103f24588'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.105286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b934b48eb3f4acbb1b674e103f24588'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.106109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2b1ee601-927a-450d-9c0b-ce4483908bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b934b48eb3f4acbb1b674e103f24588'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.117098 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 975/1962] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:35 2022] DELETE /identity/v3/users/6b934b48eb3f4acbb1b674e103f24588 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:36.469356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:36.471724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:36.490376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b8bfd-a616-4dc7-ac12-4dd571451db0 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:36.555457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:36.556661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:36.557080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:36.557398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] PATH_INFO: `/v3/users/d5d966e563e14c5292cdb683b75eb7d5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:36.558598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d966e563e14c5292cdb683b75eb7d5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:36.560901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:36.613983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d966e563e14c5292cdb683b75eb7d5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.626728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d966e563e14c5292cdb683b75eb7d5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.635700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d966e563e14c5292cdb683b75eb7d5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.643914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d966e563e14c5292cdb683b75eb7d5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.644660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d966e563e14c5292cdb683b75eb7d5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.656447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 988/1963] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:36 2022] DELETE /identity/v3/users/d5d966e563e14c5292cdb683b75eb7d5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:36.658170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:36.658643 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:36.658822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:36.687679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:36.688931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:36.760799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 976/1964] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:36.772400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:36.779250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:36.801782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:36.859731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:36.861139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:36.868878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ddf93-6642-4bc2-8ab4-46d1dae98f08 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:36.874024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcda742-a856-4609-a3e0-492967f3e3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:36.874430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcda742-a856-4609-a3e0-492967f3e3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:36.874838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcda742-a856-4609-a3e0-492967f3e3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:36.875217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcda742-a856-4609-a3e0-492967f3e3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:36.882605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd3d12-6eaf-4b0c-bfdd-c1a589c3769c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:36.888058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcda742-a856-4609-a3e0-492967f3e3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:36.889586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcda742-a856-4609-a3e0-492967f3e3d4 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:36.889746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:36.890492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:36.890802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:36.891051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] PATH_INFO: `/v3/projects/04f804d8add94b159d02e800b5ecf273` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:36.892406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f804d8add94b159d02e800b5ecf273)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:36.894835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:36.900276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 989/1965] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:36.920230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:36.977019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f804d8add94b159d02e800b5ecf273', 'request_id': 'req-374f2ac8-be41-44c3-8f7e-52eba5978a75'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.986378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f804d8add94b159d02e800b5ecf273', 'request_id': 'req-374f2ac8-be41-44c3-8f7e-52eba5978a75'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:36.993825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f804d8add94b159d02e800b5ecf273', 'request_id': 'req-374f2ac8-be41-44c3-8f7e-52eba5978a75'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.013670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] The token cache is being invalidate because project 04f804d8add94b159d02e800b5ecf273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:37.013670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.014520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-374f2ac8-be41-44c3-8f7e-52eba5978a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.019224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 977/1966] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:36 2022] DELETE /identity/v3/projects/04f804d8add94b159d02e800b5ecf273 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:37.021709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3b93-fa10-427a-b49a-bbf39e074511 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.022072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3b93-fa10-427a-b49a-bbf39e074511 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.022367 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3b93-fa10-427a-b49a-bbf39e074511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.057662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-441c3b93-fa10-427a-b49a-bbf39e074511 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:37.062607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-441c3b93-fa10-427a-b49a-bbf39e074511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.155767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 990/1967] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:37.164217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598709-3b3e-45c6-a83d-fc37c795c20f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.164808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598709-3b3e-45c6-a83d-fc37c795c20f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.165428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598709-3b3e-45c6-a83d-fc37c795c20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.212450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b2598709-3b3e-45c6-a83d-fc37c795c20f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:37.213748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b2598709-3b3e-45c6-a83d-fc37c795c20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.224278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 978/1968] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:37.228309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3b93-fa10-427a-b49a-bbf39e074511 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.267594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3b93-fa10-427a-b49a-bbf39e074511 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.275698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.276396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.276682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.276964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.277983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:37.278169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.280551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:37.307714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 991/1969] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:37.309471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598709-3b3e-45c6-a83d-fc37c795c20f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.322925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598709-3b3e-45c6-a83d-fc37c795c20f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.329167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.329367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.329659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.329942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.330809 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.333458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:37.383634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.384338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 979/1970] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:37 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:37.418439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba46cd7-29c0-4f1f-b33f-9600516a090c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.418672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.420907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.556115 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.556981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.557362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.557739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.558541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.561187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:37.568767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6383d27-5d7b-431f-a3a5-313ce6f724b0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.577495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.578457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.578741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.579017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] PATH_INFO: `/v3/projects/77105406561d4deeaad4b8d05a327d1f` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.580408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=77105406561d4deeaad4b8d05a327d1f)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.582448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:37.597113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 992/1971] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:37 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:37.597113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.598698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.614366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78669324-4490-4af0-9f92-1640779076cb admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.624651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.625747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.626126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.626683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] PATH_INFO: `/v3/projects/ae6ff0d6bca84ad699dfcace2f456a7b` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.628513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6ff0d6bca84ad699dfcace2f456a7b)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.630538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:37.649921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77105406561d4deeaad4b8d05a327d1f', 'request_id': 'req-9425a046-0cd5-40ba-af05-47ed48dff89a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.658293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77105406561d4deeaad4b8d05a327d1f', 'request_id': 'req-9425a046-0cd5-40ba-af05-47ed48dff89a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.665699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77105406561d4deeaad4b8d05a327d1f', 'request_id': 'req-9425a046-0cd5-40ba-af05-47ed48dff89a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.667834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:37.681093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] The token cache is being invalidate because project 77105406561d4deeaad4b8d05a327d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:37.681655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.682368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.684363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 980/1972] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:37 2022] DELETE /identity/v3/projects/77105406561d4deeaad4b8d05a327d1f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:52:37.685509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.686953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.696364 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6ff0d6bca84ad699dfcace2f456a7b', 'request_id': 'req-6dc29026-5fdd-4087-8ef1-9a03fd67d301'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.699501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:37.706009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6ff0d6bca84ad699dfcace2f456a7b', 'request_id': 'req-6dc29026-5fdd-4087-8ef1-9a03fd67d301'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.715189 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6ff0d6bca84ad699dfcace2f456a7b', 'request_id': 'req-6dc29026-5fdd-4087-8ef1-9a03fd67d301'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.749467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] The token cache is being invalidate because project ae6ff0d6bca84ad699dfcace2f456a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:37.749668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.751321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6dc29026-5fdd-4087-8ef1-9a03fd67d301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:37.753633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 993/1973] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:37 2022] DELETE /identity/v3/projects/ae6ff0d6bca84ad699dfcace2f456a7b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:37.755087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.755553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.755840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.779204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:37.780484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.796311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425a046-0cd5-40ba-af05-47ed48dff89a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.882426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 994/1974] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:37.883808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.885269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.898408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd21f08-e8bc-463f-87e6-c4442cc2a3d8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.900442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2476ee-650a-440d-b0fb-23ab8ee08cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.901308 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2476ee-650a-440d-b0fb-23ab8ee08cda admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.901599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2476ee-650a-440d-b0fb-23ab8ee08cda admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.901876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2476ee-650a-440d-b0fb-23ab8ee08cda admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.902697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2476ee-650a-440d-b0fb-23ab8ee08cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.904733 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2476ee-650a-440d-b0fb-23ab8ee08cda admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:37.905158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.906455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.906898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.907468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] PATH_INFO: `/v3/users/059d021bdaae452bbe07ceec663b1b6f` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.909299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=059d021bdaae452bbe07ceec663b1b6f)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.912757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:37.923113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 981/1975] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:37.930702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.931140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.931867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.952047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:37.953519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.965475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 982/1976] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:37.970021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:37.971762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:37.986685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418d865-e171-4a26-896b-c85dfc67bfcd None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:37.991933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:37.994166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:37.994530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:37.994878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:37.995736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:37.997630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.006621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059d021bdaae452bbe07ceec663b1b6f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:38.009468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 983/1977] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:38.010397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.022040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402aab5-b640-4b42-a6e5-c46c7d5efd08 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.027331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.028211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.028569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.028902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.029861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:38.030466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.032039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.032318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059d021bdaae452bbe07ceec663b1b6f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:38.045195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059d021bdaae452bbe07ceec663b1b6f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:38.051983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 984/1978] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:38.052986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.057448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059d021bdaae452bbe07ceec663b1b6f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:38.058598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059d021bdaae452bbe07ceec663b1b6f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:38.067338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 995/1979] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:37 2022] DELETE /identity/v3/users/059d021bdaae452bbe07ceec663b1b6f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:38.068705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.094508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:38.108918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add7f85-eb4e-49a1-8d21-7763fe51d974 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.195400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:38.255338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6147d-e9fc-45b8-85ed-4cf980032b17 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.261215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.262387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.264203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.264568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.265793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.268957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.319134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 996/1980] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:38 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:38.322635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.340932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89c180-9786-4d33-b96e-76a5166ccf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.343964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89c180-9786-4d33-b96e-76a5166ccf2b admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.343964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89c180-9786-4d33-b96e-76a5166ccf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.344262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89c180-9786-4d33-b96e-76a5166ccf2b admin admin] PATH_INFO: `/v3/projects/79ffb07ce6d44169bd05dfcd92da802e/users/df15aa2d69eb4e0498943373f1454743/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.375702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89c180-9786-4d33-b96e-76a5166ccf2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79ffb07ce6d44169bd05dfcd92da802e, user_id=df15aa2d69eb4e0498943373f1454743, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.379851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89c180-9786-4d33-b96e-76a5166ccf2b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.404964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 985/1981] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:38 2022] PUT /identity/v3/projects/79ffb07ce6d44169bd05dfcd92da802e/users/df15aa2d69eb4e0498943373f1454743/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:38.417083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267da20-2b40-4c57-8a63-5fb5ca3d7bc5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.417389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267da20-2b40-4c57-8a63-5fb5ca3d7bc5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.417673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267da20-2b40-4c57-8a63-5fb5ca3d7bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.430886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:38.516077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956266e6-37e7-4cf6-a7c7-f5126724d9a1 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.518181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a267da20-2b40-4c57-8a63-5fb5ca3d7bc5 None None] MFA Rules not processed for user `df15aa2d69eb4e0498943373f1454743`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:38.519502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a267da20-2b40-4c57-8a63-5fb5ca3d7bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:38.614826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.615576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.615893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.616170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.617334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.619627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.667604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 986/1982] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:38.675777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 997/1983] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:38 2022] POST /identity/v3/users => generated 289 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:38.680379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267da20-2b40-4c57-8a63-5fb5ca3d7bc5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.706271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.708371 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:38.714635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267da20-2b40-4c57-8a63-5fb5ca3d7bc5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.723315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.725158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.725158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.725552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.726420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.728448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.748826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 987/1984] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:38.754480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.767622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d89de-4a40-4165-bfb0-2b3f5e58ba74 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.778143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.780668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.781059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.781416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.782240 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.783927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.797523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 988/1985] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:38.802841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.816225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8bd24-ec0c-406c-a42f-b3a3f7e40cd0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.830594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.831124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:38.832458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.832742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.833130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] PATH_INFO: `/v3/projects/5ce4d31234af4b30ad2a4a3d74bc1a79/users/f9205ac45a0249aa963716f23d42ed4f/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.838472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e8650-b795-46df-9f93-5c9f4dbd3c37 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:38.854547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f329d8-b683-40ba-b77c-43b76da5c304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:38.855732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f329d8-b683-40ba-b77c-43b76da5c304 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.856216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f329d8-b683-40ba-b77c-43b76da5c304 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.856731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f329d8-b683-40ba-b77c-43b76da5c304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.857996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce4d31234af4b30ad2a4a3d74bc1a79, user_id=f9205ac45a0249aa963716f23d42ed4f, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.861500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.874174 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f329d8-b683-40ba-b77c-43b76da5c304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:38.875769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f329d8-b683-40ba-b77c-43b76da5c304 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:38.887200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 989/1986] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:38 2022] PUT /identity/v3/projects/5ce4d31234af4b30ad2a4a3d74bc1a79/users/f9205ac45a0249aa963716f23d42ed4f/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:38.899853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:38.900391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 998/1987] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:38.905025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17374-340b-48f3-a589-0733400f514c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:38.905182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:38.906079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17374-340b-48f3-a589-0733400f514c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:38.908073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17374-340b-48f3-a589-0733400f514c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:38.978441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-72a17374-340b-48f3-a589-0733400f514c None None] MFA Rules not processed for user `f9205ac45a0249aa963716f23d42ed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:38.981151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-72a17374-340b-48f3-a589-0733400f514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:39.004854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1a78-0e1d-4a5e-ad11-a811ab402c50 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:39.010368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:39.011507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:39.011723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:39.012081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] PATH_INFO: `/v3/projects/096b2d0441cb4a48a6ef5d7ec293b13d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:39.013536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] RBAC: Authorizing `identity:delete_project(project_id=096b2d0441cb4a48a6ef5d7ec293b13d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:39.015612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:39.081774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:39.083241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '096b2d0441cb4a48a6ef5d7ec293b13d', 'request_id': 'req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.091069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '096b2d0441cb4a48a6ef5d7ec293b13d', 'request_id': 'req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.097778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '096b2d0441cb4a48a6ef5d7ec293b13d', 'request_id': 'req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.134176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] The token cache is being invalidate because project 096b2d0441cb4a48a6ef5d7ec293b13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:39.134660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.135910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.139181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 990/1988] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:38 2022] DELETE /identity/v3/projects/096b2d0441cb4a48a6ef5d7ec293b13d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:39.143746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:39.145286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:39.148762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 999/1989] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:39.161104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17374-340b-48f3-a589-0733400f514c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:39.162592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-72a17374-340b-48f3-a589-0733400f514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:39.257451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b192f6f-c8f4-4d7e-a8b5-1e555cf961df admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:39.260952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17374-340b-48f3-a589-0733400f514c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:39.264478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:39.265558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:39.265837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:39.266280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] PATH_INFO: `/v3/users/ebe15545e66c4a9e95fb3333e5f7b064` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:39.268801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe15545e66c4a9e95fb3333e5f7b064)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:39.271806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:39.328497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79338e12-44e0-4c2b-afca-cd8707e4bcca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:39.329412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79338e12-44e0-4c2b-afca-cd8707e4bcca service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:39.329903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79338e12-44e0-4c2b-afca-cd8707e4bcca service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:39.330939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79338e12-44e0-4c2b-afca-cd8707e4bcca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:39.334147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-79338e12-44e0-4c2b-afca-cd8707e4bcca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:39.348155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe15545e66c4a9e95fb3333e5f7b064'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.370401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe15545e66c4a9e95fb3333e5f7b064'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.378478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe15545e66c4a9e95fb3333e5f7b064'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.390168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe15545e66c4a9e95fb3333e5f7b064'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.391009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe15545e66c4a9e95fb3333e5f7b064'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:39.408582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 991/1990] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:39 2022] DELETE /identity/v3/users/ebe15545e66c4a9e95fb3333e5f7b064 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:39.418817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:39.439576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79338e12-44e0-4c2b-afca-cd8707e4bcca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:39.441331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79338e12-44e0-4c2b-afca-cd8707e4bcca service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:39.454559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1000/1991] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:39.919188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:39.925402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:40.018178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:40.025790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18af99-b395-45d9-af24-bd4f0942ff3b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:40.033204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:40.034263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.034651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.035031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] PATH_INFO: `/v3/projects/73ab39624af742c3b379a818865c5620` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.036391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ab39624af742c3b379a818865c5620)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:40.038627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:40.094789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ab39624af742c3b379a818865c5620', 'request_id': 'req-39b3baff-1f76-44e3-a999-543a7a95904b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:40.101499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ab39624af742c3b379a818865c5620', 'request_id': 'req-39b3baff-1f76-44e3-a999-543a7a95904b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:40.108115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ab39624af742c3b379a818865c5620', 'request_id': 'req-39b3baff-1f76-44e3-a999-543a7a95904b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:40.127811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] The token cache is being invalidate because project 73ab39624af742c3b379a818865c5620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:40.128217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:40.128935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-39b3baff-1f76-44e3-a999-543a7a95904b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:40.130601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 992/1992] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:39 2022] DELETE /identity/v3/projects/73ab39624af742c3b379a818865c5620 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:40.135877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480034a-84ba-4437-9250-497a3dc7fd24 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.136372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480034a-84ba-4437-9250-497a3dc7fd24 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.136476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480034a-84ba-4437-9250-497a3dc7fd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.154661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f480034a-84ba-4437-9250-497a3dc7fd24 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:40.155930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f480034a-84ba-4437-9250-497a3dc7fd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:40.237881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1001/1993] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:40.239984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aadf7-05b2-464c-8f57-312e1e799b54 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.240410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aadf7-05b2-464c-8f57-312e1e799b54 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.240756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aadf7-05b2-464c-8f57-312e1e799b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.266855 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-2e0aadf7-05b2-464c-8f57-312e1e799b54 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:40.268403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2e0aadf7-05b2-464c-8f57-312e1e799b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:40.279002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 993/1994] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:40.283672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480034a-84ba-4437-9250-497a3dc7fd24 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:40.294184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480034a-84ba-4437-9250-497a3dc7fd24 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:40.298821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:40.299547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.299828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.300101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.300971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:40.301284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:40.302884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:40.315290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1002/1995] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:40.319698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aadf7-05b2-464c-8f57-312e1e799b54 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:40.330878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aadf7-05b2-464c-8f57-312e1e799b54 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:40.335996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:40.336694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.336979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.337253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.337963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:40.339929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:40.379796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 994/1996] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:40 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:40.384522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:40.396766 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e40b-dc2f-456f-adfd-a6a3b90773f8 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:40.464942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:40.465932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.466392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.466683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.468426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:40.472921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:40.511271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:40.511793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1003/1997] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:40 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:40.524794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19bfe8-ae02-4dbe-965f-019d5283444d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:40.529914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:40.530812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.531171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.531580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.532584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:40.534316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:40.555878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 995/1998] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:40.559815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:40.572476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375f176-58c8-4115-b333-9d66d6513a43 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:40.582042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea48b5-cf8c-43e6-9608-542bbe89316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:40.584001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea48b5-cf8c-43e6-9608-542bbe89316e admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.585105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea48b5-cf8c-43e6-9608-542bbe89316e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.585721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea48b5-cf8c-43e6-9608-542bbe89316e admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.587026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea48b5-cf8c-43e6-9608-542bbe89316e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:40.590607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea48b5-cf8c-43e6-9608-542bbe89316e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:40.638717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1004/1999] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:40.640104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:40.655141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a8c69-151e-44f9-9504-9240566680cd admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:40.669585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:40.671233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.674917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.675455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] PATH_INFO: `/v3/projects/43a77899048745279c1d8ac34a8e815c/users/3ba28e46072b48a29103cb4d6a84db24/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.695213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a77899048745279c1d8ac34a8e815c, user_id=3ba28e46072b48a29103cb4d6a84db24, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:40.698783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:40.724979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 996/2000] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:40 2022] PUT /identity/v3/projects/43a77899048745279c1d8ac34a8e815c/users/3ba28e46072b48a29103cb4d6a84db24/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:40.735710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:40.736019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:40.736315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:40.791018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] MFA Rules not processed for user `3ba28e46072b48a29103cb4d6a84db24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:40.792354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:40.899751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:40.937486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1005/2001] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:52:40.989072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:40.990836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:41.112875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db083a-ca5b-45f8-86b3-e02642b5ae2e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:41.130736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac707035-7147-417f-b55d-2dbe289fa938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:41.132247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707035-7147-417f-b55d-2dbe289fa938 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:41.132634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707035-7147-417f-b55d-2dbe289fa938 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:41.133009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707035-7147-417f-b55d-2dbe289fa938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:41.152505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707035-7147-417f-b55d-2dbe289fa938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:41.155035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707035-7147-417f-b55d-2dbe289fa938 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:41.169515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 997/2002] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:46.261773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:46.263276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:46.339678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582fe4e-3871-4cfb-8461-488d54222edd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:46.345507 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:46.346160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:46.346451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:46.346727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] PATH_INFO: `/v3/users/df15aa2d69eb4e0498943373f1454743` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:46.348890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df15aa2d69eb4e0498943373f1454743)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:46.350617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:46.401112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df15aa2d69eb4e0498943373f1454743'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:46.412916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df15aa2d69eb4e0498943373f1454743'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:46.422054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df15aa2d69eb4e0498943373f1454743'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:46.429288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df15aa2d69eb4e0498943373f1454743'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:46.429952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df15aa2d69eb4e0498943373f1454743'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:46.435954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1006/2003] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:46 2022] DELETE /identity/v3/users/df15aa2d69eb4e0498943373f1454743 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:46.440777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:46.441323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:46.441323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:46.459189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:46.460436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:46.481750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:46.527092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 998/2004] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:46.534374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:46.535928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:46.596019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195577d-da57-4802-973f-b5277becf0a4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:46.599740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6162b1c-a325-410d-be39-dbda313312fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:46.600403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162b1c-a325-410d-be39-dbda313312fe service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:46.600745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162b1c-a325-410d-be39-dbda313312fe service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:46.600965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162b1c-a325-410d-be39-dbda313312fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:46.610807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6162b1c-a325-410d-be39-dbda313312fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:46.612294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6162b1c-a325-410d-be39-dbda313312fe service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:46.622110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1007/2005] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:46.900377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:46.902493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:46.918834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429ae18-8161-471a-a56c-a18e0bcca5fa None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:46.924738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:46.925709 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:46.925858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:46.926157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] PATH_INFO: `/v3/projects/79ffb07ce6d44169bd05dfcd92da802e` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:46.927455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=79ffb07ce6d44169bd05dfcd92da802e)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:46.929380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:46.978901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79ffb07ce6d44169bd05dfcd92da802e', 'request_id': 'req-a874e469-e0fe-4299-903c-a508c14cca8e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:46.985638 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79ffb07ce6d44169bd05dfcd92da802e', 'request_id': 'req-a874e469-e0fe-4299-903c-a508c14cca8e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:46.991836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79ffb07ce6d44169bd05dfcd92da802e', 'request_id': 'req-a874e469-e0fe-4299-903c-a508c14cca8e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:47.006118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] The token cache is being invalidate because project 79ffb07ce6d44169bd05dfcd92da802e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:47.006403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:47.007020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a874e469-e0fe-4299-903c-a508c14cca8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:47.008623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 999/2006] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:46 2022] DELETE /identity/v3/projects/79ffb07ce6d44169bd05dfcd92da802e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:47.013662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02195ac-a9cb-42ce-9d44-33d6cd93b72e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.013931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02195ac-a9cb-42ce-9d44-33d6cd93b72e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.014216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02195ac-a9cb-42ce-9d44-33d6cd93b72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.033001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b02195ac-a9cb-42ce-9d44-33d6cd93b72e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:47.034225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b02195ac-a9cb-42ce-9d44-33d6cd93b72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:47.040217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:47.100920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1008/2007] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:47.109216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63e32-a302-4742-a9f3-fb043b563eac None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.109507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63e32-a302-4742-a9f3-fb043b563eac None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.109791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63e32-a302-4742-a9f3-fb043b563eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.133423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9dd63e32-a302-4742-a9f3-fb043b563eac None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:47.134635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9dd63e32-a302-4742-a9f3-fb043b563eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:47.145366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1000/2008] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:47.150383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02195ac-a9cb-42ce-9d44-33d6cd93b72e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:47.160460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02195ac-a9cb-42ce-9d44-33d6cd93b72e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:47.165277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:47.165944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.166226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.166503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.167251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:47.167588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:47.169198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:47.181288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1009/2009] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:52:47.185837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63e32-a302-4742-a9f3-fb043b563eac None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:47.195474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63e32-a302-4742-a9f3-fb043b563eac None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:47.200515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:47.201176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.201454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.201735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.202664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:47.204676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:47.235103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1001/2010] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:47 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:47.239164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:47.249943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea6ace-19cd-4d49-999b-f79a6265c64b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:47.308499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:47.309164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.309461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.309848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.310573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:47.312691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:47.336430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1010/2011] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:47 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:47.338045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:47.350060 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad07b24-4e71-483c-9408-292d202ced70 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:47.355675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:47.356362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.356649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.356929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.357669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:47.359295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:47.370867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1002/2012] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:47.374869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:47.385533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49608082-e817-4cf6-b81b-8d38076e1123 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:47.396576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019e4a3-6f40-416f-9032-6d22a0222e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:47.396576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019e4a3-6f40-416f-9032-6d22a0222e95 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.396576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019e4a3-6f40-416f-9032-6d22a0222e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.396576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019e4a3-6f40-416f-9032-6d22a0222e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.397019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019e4a3-6f40-416f-9032-6d22a0222e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:47.397019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019e4a3-6f40-416f-9032-6d22a0222e95 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:47.406758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1011/2013] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:47.411152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:47.421615 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c277a2-b0e5-4e79-aa88-1e02dc3e12fb admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:47.426364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:47.427262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.427659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.427944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] PATH_INFO: `/v3/projects/fd3e386b03924805aea83ba593c80d27/users/fafe1438091c4f6db0c7087a8b215127/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.442725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3e386b03924805aea83ba593c80d27, user_id=fafe1438091c4f6db0c7087a8b215127, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:47.445446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:47.461604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1003/2014] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:47 2022] PUT /identity/v3/projects/fd3e386b03924805aea83ba593c80d27/users/fafe1438091c4f6db0c7087a8b215127/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:47.469148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db397df-edbe-40c6-8295-e036017b0a93 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.469444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db397df-edbe-40c6-8295-e036017b0a93 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.469726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db397df-edbe-40c6-8295-e036017b0a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.488517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0db397df-edbe-40c6-8295-e036017b0a93 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:47.489727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0db397df-edbe-40c6-8295-e036017b0a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:47.553026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1012/2015] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:47.563436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:47.565304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:47.616991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:47.629573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf533975-b62b-4025-b360-36580a724cc1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:47.636218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:47.637122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:47.637486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:47.637908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:47.648836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:47.650311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:47.662197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1004/2016] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:48.312765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d746771-f04c-4404-9dd8-37f3d9961e1f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:48.313383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d746771-f04c-4404-9dd8-37f3d9961e1f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:48.313383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d746771-f04c-4404-9dd8-37f3d9961e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:48.342185 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1d746771-f04c-4404-9dd8-37f3d9961e1f None None] MFA Rules not processed for user `fafe1438091c4f6db0c7087a8b215127`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:48.343852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1d746771-f04c-4404-9dd8-37f3d9961e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:48.355981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:48.426002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1013/2017] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:48.434265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:48.436023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:48.502461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335d5c-9c85-4dfe-984e-c802c0dce895 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:48.506096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:48.506777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:48.507058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:48.507332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:48.515947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:48.517395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:48.526215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1005/2018] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:49.896548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbc195-01cf-41f5-b992-18600914eee0 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:49.897188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbc195-01cf-41f5-b992-18600914eee0 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:49.897188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbc195-01cf-41f5-b992-18600914eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:49.916936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-48bbc195-01cf-41f5-b992-18600914eee0 None None] MFA Rules not processed for user `fafe1438091c4f6db0c7087a8b215127`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:49.918140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-48bbc195-01cf-41f5-b992-18600914eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:49.925504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1014/2019] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:50.542318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:50.543932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:50.561675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61576-fd4a-41b2-a416-029aa2f0c998 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:50.573528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:50.575789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:50.576083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:50.578833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] PATH_INFO: `/v3/users/3ba28e46072b48a29103cb4d6a84db24` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:50.580736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba28e46072b48a29103cb4d6a84db24)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:50.584225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:50.691188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba28e46072b48a29103cb4d6a84db24'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:50.708747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba28e46072b48a29103cb4d6a84db24'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:50.718136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba28e46072b48a29103cb4d6a84db24'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:50.728576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba28e46072b48a29103cb4d6a84db24'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:50.729899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba28e46072b48a29103cb4d6a84db24'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:50.742020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1006/2020] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:52:50 2022] DELETE /identity/v3/users/3ba28e46072b48a29103cb4d6a84db24 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:50.746245 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:50.746627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:50.747091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:50.773032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:50.774572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:50.862629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1015/2021] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:50.872738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:50.874420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:50.947903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf430a-7022-48bd-977c-5ad904442394 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:50.951716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b0e02-0449-46d0-9804-947836f676d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:50.952519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b0e02-0449-46d0-9804-947836f676d4 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:50.952628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b0e02-0449-46d0-9804-947836f676d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:50.952970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b0e02-0449-46d0-9804-947836f676d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:50.963307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b0e02-0449-46d0-9804-947836f676d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:50.964888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b0e02-0449-46d0-9804-947836f676d4 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:50.976178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1007/2022] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:52:51.275655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.277143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:51.290628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d5f72-9758-4ede-8586-70a92bc0ddbd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:51.296793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:51.298426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.299407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.299868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] PATH_INFO: `/v3/projects/43a77899048745279c1d8ac34a8e815c` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.301116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a77899048745279c1d8ac34a8e815c)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:51.304160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:51.371940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a77899048745279c1d8ac34a8e815c', 'request_id': 'req-cccb41ec-d82b-4ad0-bd2b-656d7732860a'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:51.378881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a77899048745279c1d8ac34a8e815c', 'request_id': 'req-cccb41ec-d82b-4ad0-bd2b-656d7732860a'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:51.387084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a77899048745279c1d8ac34a8e815c', 'request_id': 'req-cccb41ec-d82b-4ad0-bd2b-656d7732860a'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:51.401691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] The token cache is being invalidate because project 43a77899048745279c1d8ac34a8e815c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:52:51.402058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:51.402690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cccb41ec-d82b-4ad0-bd2b-656d7732860a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:52:51.404329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1016/2023] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:52:51 2022] DELETE /identity/v3/projects/43a77899048745279c1d8ac34a8e815c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:51.409675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e05da1-e159-4f6f-a17d-48f8fbbdb8d4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.410038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e05da1-e159-4f6f-a17d-48f8fbbdb8d4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.410414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e05da1-e159-4f6f-a17d-48f8fbbdb8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.419215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:51.432772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-71e05da1-e159-4f6f-a17d-48f8fbbdb8d4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:51.434251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-71e05da1-e159-4f6f-a17d-48f8fbbdb8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:51.500054 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1008/2024] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:51.508741 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788fea-0384-4d19-ab8d-b8c1c9620a69 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.509113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788fea-0384-4d19-ab8d-b8c1c9620a69 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.509613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788fea-0384-4d19-ab8d-b8c1c9620a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.528805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8e788fea-0384-4d19-ab8d-b8c1c9620a69 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:51.530091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8e788fea-0384-4d19-ab8d-b8c1c9620a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:51.538743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1017/2025] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:51.543061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e05da1-e159-4f6f-a17d-48f8fbbdb8d4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.553456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e05da1-e159-4f6f-a17d-48f8fbbdb8d4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:51.558027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:51.558696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.558998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.559262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.560055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:52:51.560354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:51.561996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:51.573506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1009/2026] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:52:51.577879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788fea-0384-4d19-ab8d-b8c1c9620a69 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.587433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788fea-0384-4d19-ab8d-b8c1c9620a69 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:51.592592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:51.593282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.593911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.594199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.594899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:51.596964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:51.612603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:51.632281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1018/2027] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:51 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:51.636940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.647359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936f290-31b0-4a25-b5fa-7a4fc42e11e0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:51.719553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:51.719553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.719553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.719873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.720495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:51.722529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:51.743582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1010/2028] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:51 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:51.747792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.758419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d894c90-c436-4182-93f7-14e1bb4edb61 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:51.763860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:51.764528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.764811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.765082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.765938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:51.767687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:51.780190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1019/2029] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:51.784806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.795988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d73b3-0e2c-47b6-b17d-21046f621792 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:51.803333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872ead4-10aa-4790-ae38-a340dbe24af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:51.804056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872ead4-10aa-4790-ae38-a340dbe24af0 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.804342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872ead4-10aa-4790-ae38-a340dbe24af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.804624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872ead4-10aa-4790-ae38-a340dbe24af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.805361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872ead4-10aa-4790-ae38-a340dbe24af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:51.806982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872ead4-10aa-4790-ae38-a340dbe24af0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:51.821460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1011/2030] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:51.822840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.833611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf215ca-9c0d-474c-a1bb-7041ec1b575e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:51.838644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:51.839590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.839935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.840171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] PATH_INFO: `/v3/projects/3b5f6238effd43d39cbf0af0f4c582f8/users/109a46a7108e48c48407681b3e5b1e0c/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.857118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5f6238effd43d39cbf0af0f4c582f8, user_id=109a46a7108e48c48407681b3e5b1e0c, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:51.859834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:51.874195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1020/2031] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:51 2022] PUT /identity/v3/projects/3b5f6238effd43d39cbf0af0f4c582f8/users/109a46a7108e48c48407681b3e5b1e0c/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:51.879610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8e56-a9e2-4097-8d3a-d2e46e02edc3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:51.879903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8e56-a9e2-4097-8d3a-d2e46e02edc3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:51.880183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8e56-a9e2-4097-8d3a-d2e46e02edc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:51.898681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-fdcc8e56-a9e2-4097-8d3a-d2e46e02edc3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:51.899922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fdcc8e56-a9e2-4097-8d3a-d2e46e02edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:51.962420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1012/2032] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:51.973273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:51.975484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:52.051201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf577378-ab6a-488a-a631-7cfcc58d8f00 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:52.055212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:52.055975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:52.056410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:52.056685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:52.067721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:52.069198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:52.080074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1021/2033] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:52:56.462819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d762959-8d09-4f6e-97c3-93adb651408a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:56.463275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d762959-8d09-4f6e-97c3-93adb651408a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:56.463409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d762959-8d09-4f6e-97c3-93adb651408a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:56.488010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:52:56.500798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8d762959-8d09-4f6e-97c3-93adb651408a None None] MFA Rules not processed for user `109a46a7108e48c48407681b3e5b1e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:52:56.502319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8d762959-8d09-4f6e-97c3-93adb651408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:52:56.605981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1013/2034] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:52:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:52:56.610580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:56.622495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9756d03-891c-4bae-8dd9-9f430aa5e161 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:56.627302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:56.628094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:56.629125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:56.629392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:56.630110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:56.632438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:56.663987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1022/2035] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:52:56 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:52:56.668510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d762959-8d09-4f6e-97c3-93adb651408a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:56.683296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d762959-8d09-4f6e-97c3-93adb651408a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:56.753871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:56.754614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:56.754978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:56.755500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:56.756867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:56.759185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:56.788813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1014/2036] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:52:56 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:52:56.792454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:56.804893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d8017-41e5-49a9-bdc6-f9fffc880623 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:56.813170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:56.813983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:56.814954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:56.815323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:56.817937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:56.819820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:56.838288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1023/2037] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:52:56.844877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:56.860423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169376b-0708-4914-b3f1-b774577ccec4 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:56.868205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:56.870389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:56.870389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:56.870389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] PATH_INFO: `/v3/projects/df692fbae8174caf80369f4e4f5832c2/users/63c51101d06444218eb60b017ee40f78/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:56.906988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df692fbae8174caf80369f4e4f5832c2, user_id=63c51101d06444218eb60b017ee40f78, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:56.912099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:56.939331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1015/2038] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:56 2022] PUT /identity/v3/projects/df692fbae8174caf80369f4e4f5832c2/users/63c51101d06444218eb60b017ee40f78/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:52:56.941899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:56.954013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce8211-d6a7-4297-9977-efec542fb473 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:57.062139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:57.062139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:57.062139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:57.062139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:57.062614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:57.063704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:57.081169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1024/2039] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:52:57.085262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:52:57.100450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd0501-b91c-4dd1-946e-2302a8d1151a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:52:57.107427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a584e6-aa13-45a7-bfbf-f2fce2406d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:52:57.108187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a584e6-aa13-45a7-bfbf-f2fce2406d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:52:57.108744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a584e6-aa13-45a7-bfbf-f2fce2406d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:52:57.108966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a584e6-aa13-45a7-bfbf-f2fce2406d21 admin admin] PATH_INFO: `/v3/projects/df692fbae8174caf80369f4e4f5832c2/users/63c51101d06444218eb60b017ee40f78/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:52:57.112089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a584e6-aa13-45a7-bfbf-f2fce2406d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df692fbae8174caf80369f4e4f5832c2, user_id=63c51101d06444218eb60b017ee40f78, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:52:57.115783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a584e6-aa13-45a7-bfbf-f2fce2406d21 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:52:57.132033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1016/2040] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:52:57 2022] PUT /identity/v3/projects/df692fbae8174caf80369f4e4f5832c2/users/63c51101d06444218eb60b017ee40f78/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:53:01.432657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:01.437127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:01.458845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e5c61-7858-4a75-aa6d-4197ea51bf80 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:01.501823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:01.574802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:01.575471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:01.575763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:01.576043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] PATH_INFO: `/v3/users/f9205ac45a0249aa963716f23d42ed4f` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:01.577396 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9205ac45a0249aa963716f23d42ed4f)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:01.579391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:01.701753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9205ac45a0249aa963716f23d42ed4f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:01.727259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9205ac45a0249aa963716f23d42ed4f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:01.748176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9205ac45a0249aa963716f23d42ed4f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:01.770217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9205ac45a0249aa963716f23d42ed4f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:01.772006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9205ac45a0249aa963716f23d42ed4f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:01.789561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1025/2041] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:53:01 2022] DELETE /identity/v3/users/f9205ac45a0249aa963716f23d42ed4f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:01.794354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:01.795206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:01.795670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:01.849453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:01.857071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:01.961820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:01.994247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1017/2042] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:02.015122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:02.015122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:02.185098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4134f29-4ea6-44e1-8322-f22a8139e379 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:02.189792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3946f1c-a2f7-4b23-ae2e-bb027fd7abdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:02.191489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946f1c-a2f7-4b23-ae2e-bb027fd7abdf service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:02.191809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946f1c-a2f7-4b23-ae2e-bb027fd7abdf service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:02.192264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946f1c-a2f7-4b23-ae2e-bb027fd7abdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:02.210000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3946f1c-a2f7-4b23-ae2e-bb027fd7abdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:02.211670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3946f1c-a2f7-4b23-ae2e-bb027fd7abdf service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:02.223852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1026/2043] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:53:03.247434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:03.248884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:03.264562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e30a40-c2fd-49ae-99af-0e56ab226f4f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:03.279266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:03.280468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:03.280934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:03.281943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] PATH_INFO: `/v3/projects/5ce4d31234af4b30ad2a4a3d74bc1a79` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:03.283929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce4d31234af4b30ad2a4a3d74bc1a79)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:03.287813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:03.417274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce4d31234af4b30ad2a4a3d74bc1a79', 'request_id': 'req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:03.428028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce4d31234af4b30ad2a4a3d74bc1a79', 'request_id': 'req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:03.438854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce4d31234af4b30ad2a4a3d74bc1a79', 'request_id': 'req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:03.478884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] The token cache is being invalidate because project 5ce4d31234af4b30ad2a4a3d74bc1a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:53:03.479184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:03.483203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-6ddf3e06-ef32-4f62-b0f7-181f3ce0b4ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:03.492264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1018/2044] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:53:03 2022] DELETE /identity/v3/projects/5ce4d31234af4b30ad2a4a3d74bc1a79 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:03.494132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d65a3-3257-4545-b804-6f98d2e7f9d1 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:03.494620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d65a3-3257-4545-b804-6f98d2e7f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:03.494942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d65a3-3257-4545-b804-6f98d2e7f9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:03.529492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-6a4d65a3-3257-4545-b804-6f98d2e7f9d1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:03.530885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6a4d65a3-3257-4545-b804-6f98d2e7f9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:03.642973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:03.690417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1027/2045] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:53:03.696702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b445c-206a-473f-aa1f-da8c9ee4455c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:03.697395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b445c-206a-473f-aa1f-da8c9ee4455c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:03.697858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b445c-206a-473f-aa1f-da8c9ee4455c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:03.754582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-252b445c-206a-473f-aa1f-da8c9ee4455c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:03.759322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-252b445c-206a-473f-aa1f-da8c9ee4455c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:03.785747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1019/2046] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:03.790280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d65a3-3257-4545-b804-6f98d2e7f9d1 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:03.809579 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d65a3-3257-4545-b804-6f98d2e7f9d1 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:03.826986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:03.827845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:03.828577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:03.828755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:03.829617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:53:03.830884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:03.833504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:03.868566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1028/2047] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:53:03.869789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b445c-206a-473f-aa1f-da8c9ee4455c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:03.880692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b445c-206a-473f-aa1f-da8c9ee4455c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:03.886090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:03.886960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:03.887468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:03.887893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:03.888759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:03.890898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:03.931930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1020/2048] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:53:03 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:53:03.932612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:03.948846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32577b88-7399-4e36-90e8-dccbd0e45180 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:04.071799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:04.072675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:04.074726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:04.075111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:04.078311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:04.080465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:04.112544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1029/2049] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:53:03 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:53:04.124638 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:04.138721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3be728-41aa-44ec-a87f-32bb67e3a8bf admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:04.152286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:04.154330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:04.154783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:04.156077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:04.157031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:04.159504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:04.179793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1021/2050] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:53:04.180755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:04.198938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634df38-79d8-417f-adf3-8ca011f44416 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:04.211965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcb5d9-3a74-4566-bc92-063e75fb5b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:04.213491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcb5d9-3a74-4566-bc92-063e75fb5b15 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:04.215689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcb5d9-3a74-4566-bc92-063e75fb5b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:04.216082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcb5d9-3a74-4566-bc92-063e75fb5b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:04.217857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcb5d9-3a74-4566-bc92-063e75fb5b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:04.223919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcb5d9-3a74-4566-bc92-063e75fb5b15 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:04.257752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1030/2051] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:53:04.259173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:04.274884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541500e4-7a41-4c0f-8571-9d7ddb5fc281 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:04.293254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:04.294480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:04.294774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:04.295057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] PATH_INFO: `/v3/projects/6eae9ef94ebe47e7a585d24d25930a82/users/0549a0df5e204b2b97bdcf4c745042c7/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:04.354536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eae9ef94ebe47e7a585d24d25930a82, user_id=0549a0df5e204b2b97bdcf4c745042c7, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:04.362623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:04.390182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1022/2052] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:53:04 2022] PUT /identity/v3/projects/6eae9ef94ebe47e7a585d24d25930a82/users/0549a0df5e204b2b97bdcf4c745042c7/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:04.395272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d16f7-a4bc-4a32-ab5c-9cc19ffd3d27 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:04.395602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d16f7-a4bc-4a32-ab5c-9cc19ffd3d27 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:04.399626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d16f7-a4bc-4a32-ab5c-9cc19ffd3d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:04.456739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c08d16f7-a4bc-4a32-ab5c-9cc19ffd3d27 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:04.459792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c08d16f7-a4bc-4a32-ab5c-9cc19ffd3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:04.598706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1031/2053] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:04.644796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:04.649021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:04.765429 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c12a8-b6d3-4801-8a05-3126cce81485 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:04.769910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:04.770969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:04.770969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:04.771324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:04.786961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:04.788738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:04.806753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1023/2054] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:53:05.799638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d94aec-7972-4854-9d30-4b8f14de580b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:05.800243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d94aec-7972-4854-9d30-4b8f14de580b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:05.800513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d94aec-7972-4854-9d30-4b8f14de580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:05.833085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-40d94aec-7972-4854-9d30-4b8f14de580b None None] MFA Rules not processed for user `63c51101d06444218eb60b017ee40f78`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:05.834442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-40d94aec-7972-4854-9d30-4b8f14de580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:05.889512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:05.977962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1032/2055] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:05 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:05.981749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:05.984710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:06.030344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:06.123795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7eb5c-4a68-4ca6-b294-807b767f06c3 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:06.130968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affdffc-40ab-4d6c-a1b3-c8420ef56280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:06.131930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affdffc-40ab-4d6c-a1b3-c8420ef56280 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:06.132220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affdffc-40ab-4d6c-a1b3-c8420ef56280 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:06.132905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affdffc-40ab-4d6c-a1b3-c8420ef56280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:06.134428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5affdffc-40ab-4d6c-a1b3-c8420ef56280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:06.223185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affdffc-40ab-4d6c-a1b3-c8420ef56280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:06.225416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affdffc-40ab-4d6c-a1b3-c8420ef56280 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:06.251425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1024/2056] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:53:06.606926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da77ade-55ef-466b-8aa4-95f6c28b25b4 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:06.607263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da77ade-55ef-466b-8aa4-95f6c28b25b4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:06.607739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da77ade-55ef-466b-8aa4-95f6c28b25b4 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:06.609240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1033/2057] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:53:06.623108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:06.623708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:06.624164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:06.659310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] MFA Rules not processed for user `e34865acbc294b3b851bcea0cf6b442e`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:06.661005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:06.801800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1025/2058] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:53:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:06.819930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da77ade-55ef-466b-8aa4-95f6c28b25b4 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:06.822774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8da77ade-55ef-466b-8aa4-95f6c28b25b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:06.826128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:06.827622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:06.845967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4b6c-38b4-4963-aa78-31f5ec6245f7 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:06.860708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da77ade-55ef-466b-8aa4-95f6c28b25b4 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:06.862918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:06.863707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:06.863995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:06.864275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:06.868152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:06.877938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:06.878270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:06.878883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] PATH_INFO: `/v3/users/a17b8a752d78431dbde11be6d0812de3` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:06.887843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:06.889511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:06.891014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17b8a752d78431dbde11be6d0812de3)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:06.893322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:06.905712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1026/2059] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:53:07.031773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17b8a752d78431dbde11be6d0812de3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.053431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17b8a752d78431dbde11be6d0812de3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.089005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17b8a752d78431dbde11be6d0812de3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.100823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17b8a752d78431dbde11be6d0812de3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.101843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fef25a7f-7bf3-4cdd-8cbe-5e715b0eca27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17b8a752d78431dbde11be6d0812de3'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.131234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1034/2060] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:53:06 2022] DELETE /identity/v3/users/a17b8a752d78431dbde11be6d0812de3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:07.539798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:07.557555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a6bd-0826-4861-863d-1553a3a85c2d service placement] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:07.667956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:07.727130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:07.728505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:07.728786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:07.729066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] PATH_INFO: `/v3/projects/652fca4387ef47078860b964334a8651` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:07.731718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=652fca4387ef47078860b964334a8651)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:07.734700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:07.837548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652fca4387ef47078860b964334a8651', 'request_id': 'req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.849820 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652fca4387ef47078860b964334a8651', 'request_id': 'req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.874349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652fca4387ef47078860b964334a8651', 'request_id': 'req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.902142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] The token cache is being invalidate because project 652fca4387ef47078860b964334a8651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:53:07.902802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.903889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-f7e37279-a773-43c0-9ccd-0d8df67a8cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:07.908496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1027/2061] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:53:07 2022] DELETE /identity/v3/projects/652fca4387ef47078860b964334a8651 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:07.913087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a10d2-4332-4824-8d69-cb8f0bc834cc None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:07.913538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a10d2-4332-4824-8d69-cb8f0bc834cc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:07.913672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a10d2-4332-4824-8d69-cb8f0bc834cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:07.948193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-933a10d2-4332-4824-8d69-cb8f0bc834cc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:07.950276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-933a10d2-4332-4824-8d69-cb8f0bc834cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:08.040278 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1035/2062] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:08.049698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b694d4-8a4a-4c91-b023-d4c6dfb6e4d5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.049968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b694d4-8a4a-4c91-b023-d4c6dfb6e4d5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.050302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b694d4-8a4a-4c91-b023-d4c6dfb6e4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.077839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-70b694d4-8a4a-4c91-b023-d4c6dfb6e4d5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:08.082057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-70b694d4-8a4a-4c91-b023-d4c6dfb6e4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:08.100680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a10d2-4332-4824-8d69-cb8f0bc834cc None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:08.100903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1028/2063] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:08.111027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a10d2-4332-4824-8d69-cb8f0bc834cc None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:08.134783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:08.136523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.136968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.137447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.138489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:53:08.148734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:08.156836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:08.194070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1036/2064] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:53:08.200952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b694d4-8a4a-4c91-b023-d4c6dfb6e4d5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:08.216282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b694d4-8a4a-4c91-b023-d4c6dfb6e4d5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:08.226335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:08.227077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.227599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.227970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.228790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:08.231011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:08.278900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:08.279853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1029/2065] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:53:08 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:53:08.294608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ec82-7339-4c29-b636-4d477b0e5c20 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:08.377613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:08.419607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:08.420322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.420609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.420897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.421622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:08.426647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:08.486947 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1037/2066] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:53:08 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:53:08.491666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:08.506346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c36ed-edf4-48d2-8cac-1f336bd02d64 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:08.529465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:08.530282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.530575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.530921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.531700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:08.536064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:08.567093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1030/2067] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:53:08.567893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:08.602133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1bfd-3fe7-408c-acdd-e6f53b20d23e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:08.609666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49d640-91ec-4bff-9f90-746e6b06e165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:08.610639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49d640-91ec-4bff-9f90-746e6b06e165 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.610929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49d640-91ec-4bff-9f90-746e6b06e165 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.611207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49d640-91ec-4bff-9f90-746e6b06e165 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.612032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49d640-91ec-4bff-9f90-746e6b06e165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:08.614508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49d640-91ec-4bff-9f90-746e6b06e165 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:08.652892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1038/2068] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:53:08.657148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:08.685885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5c6ec-1bef-4852-b79f-2f36fc366584 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:08.707478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366300df-c0f8-4a24-b0c8-c8982845395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:08.709441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366300df-c0f8-4a24-b0c8-c8982845395e admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.709889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366300df-c0f8-4a24-b0c8-c8982845395e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.710456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366300df-c0f8-4a24-b0c8-c8982845395e admin admin] PATH_INFO: `/v3/projects/c16ddd358efa4cada4e45f75aa47a259/users/b8eecff3243f4627a15885e8f8e2dc87/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.743479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366300df-c0f8-4a24-b0c8-c8982845395e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16ddd358efa4cada4e45f75aa47a259, user_id=b8eecff3243f4627a15885e8f8e2dc87, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:08.746376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366300df-c0f8-4a24-b0c8-c8982845395e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:08.764561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1031/2069] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:53:08 2022] PUT /identity/v3/projects/c16ddd358efa4cada4e45f75aa47a259/users/b8eecff3243f4627a15885e8f8e2dc87/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:08.775266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6025e7-4fb5-4f85-b3c1-9229061e751e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.776082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6025e7-4fb5-4f85-b3c1-9229061e751e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.776799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6025e7-4fb5-4f85-b3c1-9229061e751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.830894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-dc6025e7-4fb5-4f85-b3c1-9229061e751e None None] MFA Rules not processed for user `b8eecff3243f4627a15885e8f8e2dc87`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:08.832623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-dc6025e7-4fb5-4f85-b3c1-9229061e751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:08.904952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1bd8c-193c-4f2f-b4e0-ce67d0aa9804 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:08.905494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1bd8c-193c-4f2f-b4e0-ce67d0aa9804 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:08.905922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1bd8c-193c-4f2f-b4e0-ce67d0aa9804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:08.929713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-cae1bd8c-193c-4f2f-b4e0-ce67d0aa9804 None None] MFA Rules not processed for user `109a46a7108e48c48407681b3e5b1e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:08.931748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cae1bd8c-193c-4f2f-b4e0-ce67d0aa9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:09.031301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1039/2070] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:09.033743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6025e7-4fb5-4f85-b3c1-9229061e751e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:09.053845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1032/2071] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:53:09.240799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6025e7-4fb5-4f85-b3c1-9229061e751e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:09.257778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:09.258537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:09.258764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:09.259048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:09.259850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:09.276112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:09.363558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1040/2072] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:53:09 2022] POST /identity/v3/projects => generated 373 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:53:09.367852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1bd8c-193c-4f2f-b4e0-ce67d0aa9804 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:09.384582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1bd8c-193c-4f2f-b4e0-ce67d0aa9804 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:09.393762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:09.512882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:09.514372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:09.514372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:09.514372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:09.515078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:09.517402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:09.564094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1033/2073] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:53:09 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:53:09.566619 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:09.592679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5538e-3de8-42e7-8a68-d8d00ac9822a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:09.621032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:09.621249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:09.621661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:09.622040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:09.623467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:09.629490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:09.654224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1041/2074] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:53:09.657851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:09.678948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad94b2e-bf6a-48e4-a5cb-5ae2032a26e3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:09.694492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:09.701648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:09.701988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:09.704728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] PATH_INFO: `/v3/projects/f801f15e9ebf474399e8b23f0b7ce800/users/f8b676107cbd4e048c38b789e488b310/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:09.796455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f801f15e9ebf474399e8b23f0b7ce800, user_id=f8b676107cbd4e048c38b789e488b310, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:09.809562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:09.850952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1034/2075] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:53:09 2022] PUT /identity/v3/projects/f801f15e9ebf474399e8b23f0b7ce800/users/f8b676107cbd4e048c38b789e488b310/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:09.854336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:09.875510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953408bc-eb79-46ba-8943-7cc46e3ac8d3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:09.928593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:10.078837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b237b7-f64f-4c79-9adb-df53ea4890d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:10.082250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b237b7-f64f-4c79-9adb-df53ea4890d1 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:10.085634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b237b7-f64f-4c79-9adb-df53ea4890d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:10.086158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b237b7-f64f-4c79-9adb-df53ea4890d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:10.087703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b237b7-f64f-4c79-9adb-df53ea4890d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:10.090855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b237b7-f64f-4c79-9adb-df53ea4890d1 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:10.127821 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1042/2076] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:53:10.132501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:10.151479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63436f04-eb65-4f2c-aee5-3646c2a39780 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:10.163225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:10.164835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:10.165363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:10.165865 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] PATH_INFO: `/v3/projects/f801f15e9ebf474399e8b23f0b7ce800/users/f8b676107cbd4e048c38b789e488b310/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:10.170807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f801f15e9ebf474399e8b23f0b7ce800, user_id=f8b676107cbd4e048c38b789e488b310, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:10.174190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:10.193678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1035/2077] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:53:10 2022] PUT /identity/v3/projects/f801f15e9ebf474399e8b23f0b7ce800/users/f8b676107cbd4e048c38b789e488b310/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:10.205465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8acd5-898e-493b-93c4-ed887526598e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:10.205803 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8acd5-898e-493b-93c4-ed887526598e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:10.206140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8acd5-898e-493b-93c4-ed887526598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:10.249404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-85e8acd5-898e-493b-93c4-ed887526598e None None] MFA Rules not processed for user `f8b676107cbd4e048c38b789e488b310`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:10.251112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-85e8acd5-898e-493b-93c4-ed887526598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:10.368640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1043/2078] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:10.377689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:10.380942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:10.509717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedd253-306d-45bd-bd29-da5b16bf745c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:10.515044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6152de-afaa-4436-9732-e3bb75f58a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:10.516350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6152de-afaa-4436-9732-e3bb75f58a6c service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:10.517184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6152de-afaa-4436-9732-e3bb75f58a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:10.517891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6152de-afaa-4436-9732-e3bb75f58a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:10.545291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6152de-afaa-4436-9732-e3bb75f58a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:10.546973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6152de-afaa-4436-9732-e3bb75f58a6c service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:10.590095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1036/2079] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:53:10.723083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8acd5-898e-493b-93c4-ed887526598e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:10.736515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8acd5-898e-493b-93c4-ed887526598e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:10.747565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:10.748311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:10.748603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:10.748882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:10.887561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:10.889457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:10.913415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1044/2080] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:53:12.635725 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616af924-81ee-469d-972b-1cba622fcab5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:12.636771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616af924-81ee-469d-972b-1cba622fcab5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:12.637209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616af924-81ee-469d-972b-1cba622fcab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:12.686306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-616af924-81ee-469d-972b-1cba622fcab5 None None] MFA Rules not processed for user `0549a0df5e204b2b97bdcf4c745042c7`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:12.695959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-616af924-81ee-469d-972b-1cba622fcab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:12.786377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:12.821873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1037/2081] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:12.826152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:12.827946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:12.883222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:12.912649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5dab5-c6b5-4d37-993f-cc041aeceddc service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:12.920809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:12.923001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:12.923346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:12.923805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:12.924699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:12.928613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:12.991186 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1045/2082] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:53:12 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:53:12.995210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616af924-81ee-469d-972b-1cba622fcab5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:13.006192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616af924-81ee-469d-972b-1cba622fcab5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:13.072104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:13.072837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:13.073121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:13.073401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:13.074127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:13.076169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:13.100297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1038/2083] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:53:12 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:53:13.103968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:13.118951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf755eda-ebe9-40b7-9a80-533bc650def3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:13.128114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:13.129204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:13.129600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:13.130042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:13.131232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:13.133192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:13.147119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1046/2084] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:53:13.154277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:13.164803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26d34-f552-48c6-bb64-1563fd846175 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:13.169734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:13.170806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:13.171094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:13.171410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] PATH_INFO: `/v3/projects/372932232b17459b8f39b39a9b55100a/users/d89a797e40164eff97d1696cab44be6e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:13.187675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=372932232b17459b8f39b39a9b55100a, user_id=d89a797e40164eff97d1696cab44be6e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:13.190370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:13.205167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1039/2085] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:53:13 2022] PUT /identity/v3/projects/372932232b17459b8f39b39a9b55100a/users/d89a797e40164eff97d1696cab44be6e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:13.209108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:13.220451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c90692-9c1b-4983-be8a-33224f1a988d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:13.282587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9427f8-1c1a-48fa-864b-af4fb9881410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:13.283277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9427f8-1c1a-48fa-864b-af4fb9881410 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:13.283610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9427f8-1c1a-48fa-864b-af4fb9881410 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:13.283895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9427f8-1c1a-48fa-864b-af4fb9881410 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:13.284636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9427f8-1c1a-48fa-864b-af4fb9881410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:13.286450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9427f8-1c1a-48fa-864b-af4fb9881410 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:13.300398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1047/2086] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:53:13.301758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:13.313639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04dbb1-c35a-4440-847c-69af7cd893ab admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:13.318562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:13.319537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:13.319827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:13.320109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] PATH_INFO: `/v3/projects/372932232b17459b8f39b39a9b55100a/users/d89a797e40164eff97d1696cab44be6e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:13.322427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372932232b17459b8f39b39a9b55100a, user_id=d89a797e40164eff97d1696cab44be6e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:13.325202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:13.337333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1040/2087] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:53:13 2022] PUT /identity/v3/projects/372932232b17459b8f39b39a9b55100a/users/d89a797e40164eff97d1696cab44be6e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:53:18.726228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79bfca-79b4-4f80-87c5-4720ae98fad7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:18.726713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79bfca-79b4-4f80-87c5-4720ae98fad7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:18.726836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79bfca-79b4-4f80-87c5-4720ae98fad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:18.772103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ad79bfca-79b4-4f80-87c5-4720ae98fad7 None None] MFA Rules not processed for user `d89a797e40164eff97d1696cab44be6e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:18.773788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ad79bfca-79b4-4f80-87c5-4720ae98fad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:18.945646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1048/2088] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:18.962807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:18.966967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:19.113454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3f463-39d7-49e0-bfc3-247b02227399 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:19.121719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960baf3-a7f6-492c-a556-f4cc52b8224b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:19.125996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960baf3-a7f6-492c-a556-f4cc52b8224b service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:19.126337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960baf3-a7f6-492c-a556-f4cc52b8224b service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:19.126616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960baf3-a7f6-492c-a556-f4cc52b8224b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:19.186553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:19.237622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960baf3-a7f6-492c-a556-f4cc52b8224b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:19.239342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960baf3-a7f6-492c-a556-f4cc52b8224b service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:19.250535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1041/2089] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:53:19.888015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:19.888449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:19.888560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:19.919744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] MFA Rules not processed for user `fafe1438091c4f6db0c7087a8b215127`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:19.921193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:19.970232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:20.036120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1049/2090] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:29.246251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=9 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:30.015352 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=9 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:48.918882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554165de-71b5-4ef6-a5c7-65ca4174c895 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:48.920377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554165de-71b5-4ef6-a5c7-65ca4174c895 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:48.921052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554165de-71b5-4ef6-a5c7-65ca4174c895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:48.943214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-554165de-71b5-4ef6-a5c7-65ca4174c895 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:48.944733 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-554165de-71b5-4ef6-a5c7-65ca4174c895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:49.005301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1042/2091] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:53:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:49.050250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:49.051899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:49.145138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f8f38-8bb3-4f09-944a-f7f2f1006c7f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:49.150970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:49.151549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:49.151837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:49.152119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:49.161881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:49.163517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:49.173882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1050/2092] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 12:53:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 12:53:49.269216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45663-73e8-41da-ad00-421d425672e8 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:49.269758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45663-73e8-41da-ad00-421d425672e8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:49.270209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45663-73e8-41da-ad00-421d425672e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:49.290830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0dc45663-73e8-41da-ad00-421d425672e8 None None] MFA Rules not processed for user `0549a0df5e204b2b97bdcf4c745042c7`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:49.292196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0dc45663-73e8-41da-ad00-421d425672e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:49.313059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1043/2093] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:57.216167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:57.342568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f904b-5103-4a82-81f0-bca8d2398596 service swift] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:57.375921 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:57.378175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:57.379317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:57.379706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] PATH_INFO: `/v3/users/b8eecff3243f4627a15885e8f8e2dc87` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:57.407456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8eecff3243f4627a15885e8f8e2dc87)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:57.410080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:57.531250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8eecff3243f4627a15885e8f8e2dc87'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:57.565721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8eecff3243f4627a15885e8f8e2dc87'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:57.585141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8eecff3243f4627a15885e8f8e2dc87'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:57.605200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8eecff3243f4627a15885e8f8e2dc87'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:57.605935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8eecff3243f4627a15885e8f8e2dc87'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:57.657369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1051/2094] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:53:57 2022] DELETE /identity/v3/users/b8eecff3243f4627a15885e8f8e2dc87 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:57.659207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:57.659761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:57.659853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:57.795275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:53:57.796600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:58.013058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1044/2095] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:53:58.053480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:58.055293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:58.085864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:58.379138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f29da-9399-4f49-8b3f-893e7386db96 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:58.384982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:58.385672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:58.385949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:58.386884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:58.427304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:58.430632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:58.453006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1052/2096] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:53:59.170077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:59.173886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:53:59.194053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc23440-1d62-437f-a591-03c3c0ffaf56 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:59.206063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:59.206920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:59.207646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:59.208073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] PATH_INFO: `/v3/projects/c16ddd358efa4cada4e45f75aa47a259` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:59.209614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16ddd358efa4cada4e45f75aa47a259)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:59.212448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:59.255915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:59.281228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16ddd358efa4cada4e45f75aa47a259', 'request_id': 'req-1351c374-bdc3-4731-a378-0e566924dcce'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.288374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16ddd358efa4cada4e45f75aa47a259', 'request_id': 'req-1351c374-bdc3-4731-a378-0e566924dcce'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.294500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16ddd358efa4cada4e45f75aa47a259', 'request_id': 'req-1351c374-bdc3-4731-a378-0e566924dcce'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.315946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] The token cache is being invalidate because project c16ddd358efa4cada4e45f75aa47a259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:53:59.316297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.317521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.319502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1045/2097] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:53:59 2022] DELETE /identity/v3/projects/c16ddd358efa4cada4e45f75aa47a259 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:59.324573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:59.335521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7f9fe-0b71-4ab2-abbe-3e1e419ba959 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:53:59.426027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:53:59.427344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:53:59.427734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:53:59.428080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] PATH_INFO: `/v3/users/f8b676107cbd4e048c38b789e488b310` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:53:59.429404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b676107cbd4e048c38b789e488b310)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:53:59.431596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:53:59.482810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b676107cbd4e048c38b789e488b310'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.500136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b676107cbd4e048c38b789e488b310'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.515385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b676107cbd4e048c38b789e488b310'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.519600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:53:59.531645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b676107cbd4e048c38b789e488b310'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.533098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f1fc4283-72df-46e4-a9ce-4f5fb06cb8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b676107cbd4e048c38b789e488b310'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:53:59.545679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1053/2098] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:53:59 2022] DELETE /identity/v3/users/f8b676107cbd4e048c38b789e488b310 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:53:59.987197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:53:59.988690 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:00.007180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c374-bdc3-4731-a378-0e566924dcce admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:00.117852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:00.118824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.119310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.119776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] PATH_INFO: `/v3/projects/f801f15e9ebf474399e8b23f0b7ce800` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.121081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f801f15e9ebf474399e8b23f0b7ce800)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:00.123113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:00.220409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f801f15e9ebf474399e8b23f0b7ce800', 'request_id': 'req-1c283e2d-6b75-4738-a2c6-20c7fa14de25'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:00.231100 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f801f15e9ebf474399e8b23f0b7ce800', 'request_id': 'req-1c283e2d-6b75-4738-a2c6-20c7fa14de25'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:00.238856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f801f15e9ebf474399e8b23f0b7ce800', 'request_id': 'req-1c283e2d-6b75-4738-a2c6-20c7fa14de25'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:00.267432 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] The token cache is being invalidate because project f801f15e9ebf474399e8b23f0b7ce800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:54:00.267780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:00.268341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1c283e2d-6b75-4738-a2c6-20c7fa14de25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:00.270239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1046/2099] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:53:59 2022] DELETE /identity/v3/projects/f801f15e9ebf474399e8b23f0b7ce800 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:00.275518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93e39-ddc9-4c16-97ad-6e268de60f88 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.276019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93e39-ddc9-4c16-97ad-6e268de60f88 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.276605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93e39-ddc9-4c16-97ad-6e268de60f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.327234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-37a93e39-ddc9-4c16-97ad-6e268de60f88 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:00.329180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-37a93e39-ddc9-4c16-97ad-6e268de60f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:00.424411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1054/2100] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:00.433585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de90891-a407-4cd6-a3c8-4de03b7ae919 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.433971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de90891-a407-4cd6-a3c8-4de03b7ae919 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.434241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de90891-a407-4cd6-a3c8-4de03b7ae919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.466852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9de90891-a407-4cd6-a3c8-4de03b7ae919 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:00.468140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9de90891-a407-4cd6-a3c8-4de03b7ae919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:00.481150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1047/2101] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:00.483624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a93e39-ddc9-4c16-97ad-6e268de60f88 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:00.518439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a93e39-ddc9-4c16-97ad-6e268de60f88 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:00.534348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:00.535237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.535634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.536005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.536855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:54:00.539399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:00.540511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:00.567564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1055/2102] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:54:00.568786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de90891-a407-4cd6-a3c8-4de03b7ae919 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:00.584367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de90891-a407-4cd6-a3c8-4de03b7ae919 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:00.592621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:00.594346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.594738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.595111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.595963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:00.600333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:00.669283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1048/2103] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:54:00 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:54:00.669759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:00.681994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b7d3a-fa75-4cd9-8a6d-a2a460fe9108 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:00.774869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:00.776054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.776686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.776973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.777699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:00.782177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:00.829772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1056/2104] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:54:00 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:54:00.833428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:00.857678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9169c8b-0292-4d98-93bd-edf3168d0461 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:00.879819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:00.880558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.880903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.881280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.882168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:00.888570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:00.918564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1049/2105] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:00.926217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:00.950742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb0a66-d89f-4564-ad9a-d65563c1a73a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:00.964078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983373d-df7c-4b23-9c46-823eee9855a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:00.964913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983373d-df7c-4b23-9c46-823eee9855a3 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:00.965272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983373d-df7c-4b23-9c46-823eee9855a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:00.965855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983373d-df7c-4b23-9c46-823eee9855a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:00.966422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983373d-df7c-4b23-9c46-823eee9855a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:00.969625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983373d-df7c-4b23-9c46-823eee9855a3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:01.000203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1057/2106] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:01.004082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:01.010618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:01.017581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5605cea-0c29-4e2d-9092-eeeccebdf107 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:01.025441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:01.025441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:01.025441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:01.025441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] PATH_INFO: `/v3/projects/3ffaf5fd38db4b58a54ed3af3c9ede01/users/23edc91cb67848b398c053ec09fe87c2/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:01.048631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffaf5fd38db4b58a54ed3af3c9ede01, user_id=23edc91cb67848b398c053ec09fe87c2, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:01.055286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:01.082727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1050/2107] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:54:01 2022] PUT /identity/v3/projects/3ffaf5fd38db4b58a54ed3af3c9ede01/users/23edc91cb67848b398c053ec09fe87c2/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:01.095342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7445b49-66f5-460f-86ce-dbd89fdf9d82 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:01.095570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7445b49-66f5-460f-86ce-dbd89fdf9d82 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:01.095761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7445b49-66f5-460f-86ce-dbd89fdf9d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:01.153920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b7445b49-66f5-460f-86ce-dbd89fdf9d82 None None] MFA Rules not processed for user `23edc91cb67848b398c053ec09fe87c2`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:01.155415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b7445b49-66f5-460f-86ce-dbd89fdf9d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:01.209884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:01.295153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1058/2108] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:01.303577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:01.308416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:01.452513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e3fc4-995c-44f4-95fd-61722296664a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:01.468333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:01.469733 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:01.470345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:01.470848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:01.496115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:01.498017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:01.540477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1051/2109] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:01.565386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7445b49-66f5-460f-86ce-dbd89fdf9d82 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:01.664359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:01.666007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:01.745102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ec21-e3ad-4165-90cd-696eb66c4569 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:01.753634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3b419-4aad-4e96-ba7b-612eb749ba48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:01.762297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3b419-4aad-4e96-ba7b-612eb749ba48 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:01.763570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3b419-4aad-4e96-ba7b-612eb749ba48 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:01.763952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3b419-4aad-4e96-ba7b-612eb749ba48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:01.764434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7445b49-66f5-460f-86ce-dbd89fdf9d82 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:01.769768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9ec3b419-4aad-4e96-ba7b-612eb749ba48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:01.773926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:01.774839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:01.775468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:01.775468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] PATH_INFO: `/v3/users/23edc91cb67848b398c053ec09fe87c2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:01.778251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] RBAC: Authorizing `identity:delete_user(user_id=23edc91cb67848b398c053ec09fe87c2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:01.783019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:01.895757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3b419-4aad-4e96-ba7b-612eb749ba48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:01.898394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3b419-4aad-4e96-ba7b-612eb749ba48 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:01.901267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23edc91cb67848b398c053ec09fe87c2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:01.923324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1052/2110] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:01 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:54:01.948168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23edc91cb67848b398c053ec09fe87c2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:01.967003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23edc91cb67848b398c053ec09fe87c2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:01.979509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23edc91cb67848b398c053ec09fe87c2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:01.980266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23edc91cb67848b398c053ec09fe87c2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:01.987670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1059/2111] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:54:01 2022] DELETE /identity/v3/users/23edc91cb67848b398c053ec09fe87c2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:01.992275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:01.992672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:01.993038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:02.034600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:02.035870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:02.071360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:02.147000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1053/2112] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:02.162274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:02.163883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:02.243906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1897c1-806e-4338-b629-2456f8c744be admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:02.248567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab40a3-b591-41f2-ac39-7a3bdd5f3260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:02.249360 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab40a3-b591-41f2-ac39-7a3bdd5f3260 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:02.249772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab40a3-b591-41f2-ac39-7a3bdd5f3260 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:02.249920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab40a3-b591-41f2-ac39-7a3bdd5f3260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:02.260526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:02.268765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab40a3-b591-41f2-ac39-7a3bdd5f3260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:02.272295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab40a3-b591-41f2-ac39-7a3bdd5f3260 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:02.301685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1060/2113] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:02.681924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:02.684247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:02.725967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf7cb-d166-4df5-9576-e345c3b01ead None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:02.741720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:02.743703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:02.745705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:02.747090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] PATH_INFO: `/v3/projects/3ffaf5fd38db4b58a54ed3af3c9ede01` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:02.749365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffaf5fd38db4b58a54ed3af3c9ede01)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:02.751660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:02.851073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffaf5fd38db4b58a54ed3af3c9ede01', 'request_id': 'req-466bee0d-5e46-4752-9c77-4addb495bda5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:02.862801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffaf5fd38db4b58a54ed3af3c9ede01', 'request_id': 'req-466bee0d-5e46-4752-9c77-4addb495bda5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:02.891409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffaf5fd38db4b58a54ed3af3c9ede01', 'request_id': 'req-466bee0d-5e46-4752-9c77-4addb495bda5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:02.917138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] The token cache is being invalidate because project 3ffaf5fd38db4b58a54ed3af3c9ede01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:54:02.917543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:02.918290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-466bee0d-5e46-4752-9c77-4addb495bda5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:02.923545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1054/2114] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:54:02 2022] DELETE /identity/v3/projects/3ffaf5fd38db4b58a54ed3af3c9ede01 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:02.927103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf478a-6744-4396-a741-faea27d31f37 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:02.927103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf478a-6744-4396-a741-faea27d31f37 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:02.927772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf478a-6744-4396-a741-faea27d31f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:02.949120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1daf478a-6744-4396-a741-faea27d31f37 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:02.950211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1daf478a-6744-4396-a741-faea27d31f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:03.092886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac2d9-a924-46f4-a528-999b262f80c4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.093196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac2d9-a924-46f4-a528-999b262f80c4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.093680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac2d9-a924-46f4-a528-999b262f80c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.102203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1061/2115] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:03.113230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4c3ac2d9-a924-46f4-a528-999b262f80c4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:03.114899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4c3ac2d9-a924-46f4-a528-999b262f80c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:03.127685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf478a-6744-4396-a741-faea27d31f37 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:03.133840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1055/2116] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:54:03.137702 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf478a-6744-4396-a741-faea27d31f37 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:03.143114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:03.143543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.143826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.144106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.144854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:54:03.145165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:03.146781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:03.174656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac2d9-a924-46f4-a528-999b262f80c4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:03.184440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac2d9-a924-46f4-a528-999b262f80c4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:03.185489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1062/2117] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:54:03.190995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:03.191722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.192008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.192316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.193170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:03.195105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:03.229580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1056/2118] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:54:03 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:54:03.230755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:03.243456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868c908-4aac-40fb-a02c-7adb91a4145c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:03.342109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:03.345234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.345234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.345234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.345234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:03.348086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:03.407016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1063/2119] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:54:03 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:54:03.412547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:03.434461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8930a-0fde-46ea-aa7c-9d1ad8cfb914 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:03.460774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:03.461558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.461888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.462108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.465087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:03.475011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:03.514039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1057/2120] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:03.515648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:03.538873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28912de5-ccef-4212-91dc-c1687fb5fcc0 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:03.553724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef890da9-9b7f-4601-b39a-3387bb5f508c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:03.554428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef890da9-9b7f-4601-b39a-3387bb5f508c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.554714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef890da9-9b7f-4601-b39a-3387bb5f508c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.554995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef890da9-9b7f-4601-b39a-3387bb5f508c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.555791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef890da9-9b7f-4601-b39a-3387bb5f508c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:03.557853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef890da9-9b7f-4601-b39a-3387bb5f508c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:03.615033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1064/2121] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:03.619528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:03.652721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ef19-099e-478e-ae32-d341def43658 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:03.685718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:03.691339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.691736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.692023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] PATH_INFO: `/v3/projects/1af8ff0862734b47b028a1da45dcf3a0/users/27520bf5aeed4ded9f5dc99f46dd0881/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.739266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af8ff0862734b47b028a1da45dcf3a0, user_id=27520bf5aeed4ded9f5dc99f46dd0881, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:03.747633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:03.769857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1058/2122] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:54:03 2022] PUT /identity/v3/projects/1af8ff0862734b47b028a1da45dcf3a0/users/27520bf5aeed4ded9f5dc99f46dd0881/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:54:03.785427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213840d-6272-45d4-ae03-b50c40165877 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:03.785724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213840d-6272-45d4-ae03-b50c40165877 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:03.785999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213840d-6272-45d4-ae03-b50c40165877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:03.842496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0213840d-6272-45d4-ae03-b50c40165877 None None] MFA Rules not processed for user `27520bf5aeed4ded9f5dc99f46dd0881`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:03.844234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0213840d-6272-45d4-ae03-b50c40165877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:04.008679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:04.039538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1065/2123] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:04.066588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:04.068533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:04.181139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370080-318c-44d1-af38-02a786300880 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:04.189189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:04.189875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:04.190165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:04.190956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:04.213699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:04.217971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:04.220007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:04.245619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1059/2124] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:05.166411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da4c07-c037-4ff6-b6c2-8bf7e86bd7c5 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:05.167754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da4c07-c037-4ff6-b6c2-8bf7e86bd7c5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:05.168125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da4c07-c037-4ff6-b6c2-8bf7e86bd7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:05.195790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c0da4c07-c037-4ff6-b6c2-8bf7e86bd7c5 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:05.198106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c0da4c07-c037-4ff6-b6c2-8bf7e86bd7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:05.299590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1066/2125] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:54:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:05.306512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:05.308298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:05.454044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b79f45-647a-46fb-8344-83014f547085 service glance] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:05.467072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86488c-ef6f-418c-8780-aafe37d31f69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:05.468580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86488c-ef6f-418c-8780-aafe37d31f69 service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:05.469019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86488c-ef6f-418c-8780-aafe37d31f69 service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:05.469501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86488c-ef6f-418c-8780-aafe37d31f69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:05.492518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86488c-ef6f-418c-8780-aafe37d31f69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:05.494276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86488c-ef6f-418c-8780-aafe37d31f69 service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:05.515918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1060/2126] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 12:54:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 12:54:09.042181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0be62-1704-46da-8d29-722fc34fd675 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:09.042792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0be62-1704-46da-8d29-722fc34fd675 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:09.042792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0be62-1704-46da-8d29-722fc34fd675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:09.088602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-afa0be62-1704-46da-8d29-722fc34fd675 None None] MFA Rules not processed for user `9863e406790b413faf022554ba4b8dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:09.089858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-afa0be62-1704-46da-8d29-722fc34fd675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:09.244015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1067/2127] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:10.720582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58192057-64cf-4497-907a-de46c2cb13cf None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:10.721215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58192057-64cf-4497-907a-de46c2cb13cf None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:10.721215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58192057-64cf-4497-907a-de46c2cb13cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:10.776979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-58192057-64cf-4497-907a-de46c2cb13cf None None] MFA Rules not processed for user `efee42b434cd47d99984b85f677983c2`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:10.780671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-58192057-64cf-4497-907a-de46c2cb13cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:11.020858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1061/2128] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:18.859052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:18.859686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:18.859686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:18.898123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:18.899441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:18.912368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1068/2129] 10.222.0.87 () {58 vars in 967 bytes} [Fri Jun 17 12:54:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:18.930573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58192057-64cf-4497-907a-de46c2cb13cf None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:18.948948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58192057-64cf-4497-907a-de46c2cb13cf None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:18.956582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:18.957830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:18.959245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:18.959245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:18.969331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:18.971274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:18.991157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1062/2130] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 12:54:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 12:54:21.639231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:21.641511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:21.737421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539c8e4-0851-43ba-a5fc-216dbfed5a99 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:21.750306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:21.752053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:21.752503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:21.753120 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] PATH_INFO: `/v3/users/fafe1438091c4f6db0c7087a8b215127` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:21.754719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafe1438091c4f6db0c7087a8b215127)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:21.757194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:21.807228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:21.839268 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafe1438091c4f6db0c7087a8b215127'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:21.853015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafe1438091c4f6db0c7087a8b215127'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:21.862054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafe1438091c4f6db0c7087a8b215127'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:21.869727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafe1438091c4f6db0c7087a8b215127'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:21.870461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3c4886a-dec2-4eaf-80c2-b8edda6c5bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafe1438091c4f6db0c7087a8b215127'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:21.876522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1069/2131] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:54:21 2022] DELETE /identity/v3/users/fafe1438091c4f6db0c7087a8b215127 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:22.131756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.150825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0d9e6-a3cf-40c2-8375-465693d2af79 service swift] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:22.235266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:22.237270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.237720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.237997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] PATH_INFO: `/v3/projects/fd3e386b03924805aea83ba593c80d27` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.239295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3e386b03924805aea83ba593c80d27)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:22.241455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:22.270729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:22.303259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3e386b03924805aea83ba593c80d27', 'request_id': 'req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:22.311156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3e386b03924805aea83ba593c80d27', 'request_id': 'req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:22.325612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3e386b03924805aea83ba593c80d27', 'request_id': 'req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:22.338856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] The token cache is being invalidate because project fd3e386b03924805aea83ba593c80d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:54:22.339159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:22.340014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dadebc29-d6bb-42f9-9a7e-486cb0c2ac0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:22.341720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1063/2132] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:54:22 2022] DELETE /identity/v3/projects/fd3e386b03924805aea83ba593c80d27 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:22.346951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245fd5e-8b6f-4a3e-9e82-fbc44551b071 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.347248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245fd5e-8b6f-4a3e-9e82-fbc44551b071 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.347555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245fd5e-8b6f-4a3e-9e82-fbc44551b071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.384021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4245fd5e-8b6f-4a3e-9e82-fbc44551b071 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:22.385448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4245fd5e-8b6f-4a3e-9e82-fbc44551b071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:22.460418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1070/2133] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:22.467961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244cf71-9468-4b7d-8a4a-3cd9facbf8b5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.468246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244cf71-9468-4b7d-8a4a-3cd9facbf8b5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.468675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244cf71-9468-4b7d-8a4a-3cd9facbf8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.492172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d244cf71-9468-4b7d-8a4a-3cd9facbf8b5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:22.493370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d244cf71-9468-4b7d-8a4a-3cd9facbf8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:22.507633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245fd5e-8b6f-4a3e-9e82-fbc44551b071 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.507859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1064/2134] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:22.518521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245fd5e-8b6f-4a3e-9e82-fbc44551b071 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:22.523313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:22.524022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.524308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.524570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.525312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:54:22.525635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:22.527238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:22.540833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1071/2135] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 12:54:22.545163 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244cf71-9468-4b7d-8a4a-3cd9facbf8b5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.554792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244cf71-9468-4b7d-8a4a-3cd9facbf8b5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:22.560040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:22.560723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.561009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.561286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.561995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:22.563981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:22.595502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1065/2136] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:54:22 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:54:22.597132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.613763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01c195-1af2-466e-bda8-061a6789d068 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:22.680962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:22.681658 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.681898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.682169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.682915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:22.689571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:22.717289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1072/2137] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:54:22 2022] POST /identity/v3/users => generated 285 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:54:22.721389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.733292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f12d78-a362-4ea0-9d47-58338c799a2f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:22.738660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:22.739407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.740028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.740521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.741476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:22.743416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:22.756553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1066/2138] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:22.760564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.775522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bb0f9-471e-4585-bff8-c31d7f9080f8 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:22.780756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21f01c-63fb-459f-8790-0f96fb402844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:22.782091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21f01c-63fb-459f-8790-0f96fb402844 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.782091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21f01c-63fb-459f-8790-0f96fb402844 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.782091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21f01c-63fb-459f-8790-0f96fb402844 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.782800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21f01c-63fb-459f-8790-0f96fb402844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:22.785338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21f01c-63fb-459f-8790-0f96fb402844 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:22.791201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:22.800981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1073/2139] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:22.804131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.816365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee337a-5492-4601-a4b4-bc78317b58cf admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:22.830640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:22.831602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.831886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.832165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] PATH_INFO: `/v3/projects/52d0fdeac3734a478ae6308ba21ba6d8/users/1e2d8e227f804dcc90ad527c1f73795e/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.847550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d0fdeac3734a478ae6308ba21ba6d8, user_id=1e2d8e227f804dcc90ad527c1f73795e, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:22.850581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:22.866205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1067/2140] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:54:22 2022] PUT /identity/v3/projects/52d0fdeac3734a478ae6308ba21ba6d8/users/1e2d8e227f804dcc90ad527c1f73795e/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:22.877861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:22.878264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:22.878610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:22.906276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] MFA Rules not processed for user `1e2d8e227f804dcc90ad527c1f73795e`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:22.907670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:22.984413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1074/2141] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:22.997115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:22.999160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:23.062098 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bbd97-7dc3-4a9a-866b-8e588ceb12f0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:23.066005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87993b-3c8a-4568-b14c-f468591c05da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:23.066665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87993b-3c8a-4568-b14c-f468591c05da service glance] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:23.066952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87993b-3c8a-4568-b14c-f468591c05da service glance] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:23.067226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87993b-3c8a-4568-b14c-f468591c05da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:23.075931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87993b-3c8a-4568-b14c-f468591c05da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:23.077378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87993b-3c8a-4568-b14c-f468591c05da service glance] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:23.086503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1068/2142] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:26.521478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:26.522994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:26.585282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612bc72-1f5a-47e7-ab50-c85ede7078e2 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:26.590413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490968eb-8e06-46a1-a7b8-794ccc4490f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:26.591078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490968eb-8e06-46a1-a7b8-794ccc4490f7 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:26.591363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490968eb-8e06-46a1-a7b8-794ccc4490f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:26.591988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490968eb-8e06-46a1-a7b8-794ccc4490f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:26.594253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-490968eb-8e06-46a1-a7b8-794ccc4490f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:26.662143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490968eb-8e06-46a1-a7b8-794ccc4490f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:26.663612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490968eb-8e06-46a1-a7b8-794ccc4490f7 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:26.673251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1075/2143] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:29.302285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1369-9b67-489f-8155-288a3844c1b4 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:29.302812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1369-9b67-489f-8155-288a3844c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:29.302812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1369-9b67-489f-8155-288a3844c1b4 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:29.304406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1069/2144] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:54:29.310538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821903-b4f3-4d70-b1b4-c5726d0d3114 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:29.311036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821903-b4f3-4d70-b1b4-c5726d0d3114 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:29.311148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821903-b4f3-4d70-b1b4-c5726d0d3114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:29.370834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-19821903-b4f3-4d70-b1b4-c5726d0d3114 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:29.372179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-19821903-b4f3-4d70-b1b4-c5726d0d3114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:29.386552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:29.485524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1076/2145] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:54:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:29.497848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc1369-9b67-489f-8155-288a3844c1b4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:29.499653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5bfc1369-9b67-489f-8155-288a3844c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:29.514065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc1369-9b67-489f-8155-288a3844c1b4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:29.518009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3510b1-fc4b-44be-af0f-ff7b711ad12e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:29.519006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3510b1-fc4b-44be-af0f-ff7b711ad12e service glance] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:29.519291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3510b1-fc4b-44be-af0f-ff7b711ad12e service glance] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:29.519684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3510b1-fc4b-44be-af0f-ff7b711ad12e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:29.531339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3510b1-fc4b-44be-af0f-ff7b711ad12e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:29.532881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3510b1-fc4b-44be-af0f-ff7b711ad12e service glance] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:29.545746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1070/2146] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:29.623345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9e816-a6c0-4458-b870-d315215b1633 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:29.623688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9e816-a6c0-4458-b870-d315215b1633 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:29.623953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9e816-a6c0-4458-b870-d315215b1633 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:29.625392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1077/2147] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 12:54:29.630866 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00fb1f-4dc2-4e60-a20b-4efd897a2530 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:29.631224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00fb1f-4dc2-4e60-a20b-4efd897a2530 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:29.631596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00fb1f-4dc2-4e60-a20b-4efd897a2530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:29.650889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-2d00fb1f-4dc2-4e60-a20b-4efd897a2530 None None] MFA Rules not processed for user `40f6ae0e5e164461b4c51b829be9a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:29.652119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2d00fb1f-4dc2-4e60-a20b-4efd897a2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:29.659352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1071/2148] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:54:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:29.672608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9e816-a6c0-4458-b870-d315215b1633 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:29.685064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9e816-a6c0-4458-b870-d315215b1633 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:29.688732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6556-67c7-4768-8dbc-35082a3aa430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:29.689386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6556-67c7-4768-8dbc-35082a3aa430 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:29.689675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6556-67c7-4768-8dbc-35082a3aa430 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:29.689934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6556-67c7-4768-8dbc-35082a3aa430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:29.698326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b6556-67c7-4768-8dbc-35082a3aa430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:29.699798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b6556-67c7-4768-8dbc-35082a3aa430 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:29.709313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1078/2149] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:30.068512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30befa-efb9-4c16-aabb-452968ca1174 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:30.069335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30befa-efb9-4c16-aabb-452968ca1174 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:30.069945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30befa-efb9-4c16-aabb-452968ca1174 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:30.071646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1072/2150] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 12:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 12:54:30.079139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:30.079886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:30.080503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:30.103580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:30.105055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:30.148234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1079/2151] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 12:54:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:54:30.181294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30befa-efb9-4c16-aabb-452968ca1174 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:30.185342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8f30befa-efb9-4c16-aabb-452968ca1174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:30.208982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30befa-efb9-4c16-aabb-452968ca1174 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:30.218480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57306344-b587-4819-a8ec-3c0c135c730f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:30.219177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306344-b587-4819-a8ec-3c0c135c730f service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:30.219499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306344-b587-4819-a8ec-3c0c135c730f service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:30.219851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306344-b587-4819-a8ec-3c0c135c730f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:30.234192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57306344-b587-4819-a8ec-3c0c135c730f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:30.236269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57306344-b587-4819-a8ec-3c0c135c730f service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:30.248744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1073/2152] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:32.669699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:32.673070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:32.763829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01796ad4-59b6-4a03-845b-3d396bbd9043 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:32.768841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:32.769740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:32.770023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:32.770296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] PATH_INFO: `/v3/users/27520bf5aeed4ded9f5dc99f46dd0881` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:32.771638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27520bf5aeed4ded9f5dc99f46dd0881)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:32.773697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:32.840794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27520bf5aeed4ded9f5dc99f46dd0881'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:32.856349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27520bf5aeed4ded9f5dc99f46dd0881'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:32.867577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27520bf5aeed4ded9f5dc99f46dd0881'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:32.871021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:32.887576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27520bf5aeed4ded9f5dc99f46dd0881'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:32.889317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27520bf5aeed4ded9f5dc99f46dd0881'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:32.897568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1080/2153] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:54:32 2022] DELETE /identity/v3/users/27520bf5aeed4ded9f5dc99f46dd0881 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:32.902229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:32.904328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:32.904809 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:32.929289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:32.930846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:32.940562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:33.012258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1074/2154] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:33.026323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:33.028906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:33.145287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e17df7-b587-4b40-a4cb-13954628b899 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:33.154812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:33.155840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:33.156197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:33.157112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:33.174267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:33.174267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:33.187351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1081/2155] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:33.334589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:33.336229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:33.524076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9b86f-3e05-4756-9804-fce7614a5261 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:33.534689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ca84e-6ede-4b1d-8c6c-886114a36a79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:33.535546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ca84e-6ede-4b1d-8c6c-886114a36a79 service placement] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:33.535831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ca84e-6ede-4b1d-8c6c-886114a36a79 service placement] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:33.536278 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ca84e-6ede-4b1d-8c6c-886114a36a79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:33.538827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-075ca84e-6ede-4b1d-8c6c-886114a36a79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:33.553735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ca84e-6ede-4b1d-8c6c-886114a36a79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:33.559610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ca84e-6ede-4b1d-8c6c-886114a36a79 service placement] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:33.571986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1075/2156] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:54:34.434943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:34.436641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:34.454956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af1a3d-1229-4d59-9e26-4d824c82b104 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:34.467191 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:34.468748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:34.469040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:34.469656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] PATH_INFO: `/v3/projects/1af8ff0862734b47b028a1da45dcf3a0` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:34.471108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af8ff0862734b47b028a1da45dcf3a0)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:34.477530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:34.536543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af8ff0862734b47b028a1da45dcf3a0', 'request_id': 'req-9184836e-e1e9-4004-8bfc-339f7898c913'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:34.544542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af8ff0862734b47b028a1da45dcf3a0', 'request_id': 'req-9184836e-e1e9-4004-8bfc-339f7898c913'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:34.555437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af8ff0862734b47b028a1da45dcf3a0', 'request_id': 'req-9184836e-e1e9-4004-8bfc-339f7898c913'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:34.576256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] The token cache is being invalidate because project 1af8ff0862734b47b028a1da45dcf3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:54:34.576585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:34.578807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9184836e-e1e9-4004-8bfc-339f7898c913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:34.581144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1082/2157] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:54:34 2022] DELETE /identity/v3/projects/1af8ff0862734b47b028a1da45dcf3a0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:34.586577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ef87-998d-43fd-a86f-a776ece7b7bb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:34.587096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ef87-998d-43fd-a86f-a776ece7b7bb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:34.587221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ef87-998d-43fd-a86f-a776ece7b7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:34.622696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-69d8ef87-998d-43fd-a86f-a776ece7b7bb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:34.623964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-69d8ef87-998d-43fd-a86f-a776ece7b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:34.772256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1076/2158] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:34.780596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75465c-ad99-4a94-81da-e4823918cc5a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:34.780944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75465c-ad99-4a94-81da-e4823918cc5a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:34.781148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75465c-ad99-4a94-81da-e4823918cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:34.815847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7b75465c-ad99-4a94-81da-e4823918cc5a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:34.817228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7b75465c-ad99-4a94-81da-e4823918cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:34.834406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1083/2159] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:34.835273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8ef87-998d-43fd-a86f-a776ece7b7bb None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:34.847492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8ef87-998d-43fd-a86f-a776ece7b7bb None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:34.858511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:34.859435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:34.859701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:34.860620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:34.861390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:54:34.862194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:34.863569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:34.878813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1077/2160] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:54:34.892234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75465c-ad99-4a94-81da-e4823918cc5a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:34.902345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75465c-ad99-4a94-81da-e4823918cc5a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:34.908971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:34.909202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:34.909485 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:34.909765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:34.910480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:34.912623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:34.947104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1084/2161] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:54:34 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:54:34.948101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:34.960510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e4c64-8b6b-41d2-859b-0bd608301b66 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:34.969611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:35.028932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:35.029498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.029860 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.030249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.031186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:35.033526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:35.059425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1078/2162] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:54:34 2022] POST /identity/v3/users => generated 281 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:54:35.061122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:35.076552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee9b6-39c7-4511-9562-bf063748745c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:35.081914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:35.084073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.084073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.084073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.084410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:35.086178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:35.101074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1085/2163] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:35.102320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:35.114422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b378637-6fa6-472d-95b4-07f607b49204 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:35.122391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641a9cb-0b24-4d6a-869f-476a55cab330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:35.123761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641a9cb-0b24-4d6a-869f-476a55cab330 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.124217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641a9cb-0b24-4d6a-869f-476a55cab330 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.124488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641a9cb-0b24-4d6a-869f-476a55cab330 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.125245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641a9cb-0b24-4d6a-869f-476a55cab330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:35.127037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641a9cb-0b24-4d6a-869f-476a55cab330 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:35.139419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1079/2164] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:54:35.142471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:35.162182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee91f5-e373-4aba-ab06-01eace3d3544 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:35.167820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:35.168856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.169233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.169607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] PATH_INFO: `/v3/projects/bfae378d88ba40cba43f703ec40d882b/users/6bfa2b56cc754445801c4f86a14de42e/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.193291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfae378d88ba40cba43f703ec40d882b, user_id=6bfa2b56cc754445801c4f86a14de42e, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:35.197276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:35.219461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1086/2165] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:54:35 2022] PUT /identity/v3/projects/bfae378d88ba40cba43f703ec40d882b/users/6bfa2b56cc754445801c4f86a14de42e/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:35.231662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77a745-36eb-4e04-b8de-82f5c8e291ac None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.231953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77a745-36eb-4e04-b8de-82f5c8e291ac None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.232228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77a745-36eb-4e04-b8de-82f5c8e291ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.260230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-7c77a745-36eb-4e04-b8de-82f5c8e291ac None None] MFA Rules not processed for user `6bfa2b56cc754445801c4f86a14de42e`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:35.261471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7c77a745-36eb-4e04-b8de-82f5c8e291ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:35.350162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1080/2166] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:35.383831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:35.385564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:35.410620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:35.501279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99c1ff-7c3a-4d65-8614-c41498a304c3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:35.507118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573412f-aa72-4737-b3c3-c52723297553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:35.508071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573412f-aa72-4737-b3c3-c52723297553 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.508517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573412f-aa72-4737-b3c3-c52723297553 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.509082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573412f-aa72-4737-b3c3-c52723297553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.520199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573412f-aa72-4737-b3c3-c52723297553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:35.524933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573412f-aa72-4737-b3c3-c52723297553 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:35.540017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1087/2167] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:35.557685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77a745-36eb-4e04-b8de-82f5c8e291ac None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:35.660807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77a745-36eb-4e04-b8de-82f5c8e291ac None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:35.667402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:35.667402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.667746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.669094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] PATH_INFO: `/v3/users/6bfa2b56cc754445801c4f86a14de42e` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.669751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bfa2b56cc754445801c4f86a14de42e)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:35.675654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:35.720108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:35.777441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bfa2b56cc754445801c4f86a14de42e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:35.794659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bfa2b56cc754445801c4f86a14de42e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:35.802177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bfa2b56cc754445801c4f86a14de42e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:35.812869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bfa2b56cc754445801c4f86a14de42e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:35.813761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bfa2b56cc754445801c4f86a14de42e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:35.820978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1081/2168] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:54:35 2022] DELETE /identity/v3/users/6bfa2b56cc754445801c4f86a14de42e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:35.825884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e614-192e-4b2b-bd72-52241403641a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:35.826196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e614-192e-4b2b-bd72-52241403641a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:35.826477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e614-192e-4b2b-bd72-52241403641a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:35.847525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3775e614-192e-4b2b-bd72-52241403641a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:35.848953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3775e614-192e-4b2b-bd72-52241403641a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:35.929431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1088/2169] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:35.947048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:35.951885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:36.017171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1190b9-333b-4ed5-8c90-908826058b3d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:36.022746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cd010-cd70-4932-a3e1-ae8a315534e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:36.023580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cd010-cd70-4932-a3e1-ae8a315534e9 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:36.023944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cd010-cd70-4932-a3e1-ae8a315534e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:36.024314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cd010-cd70-4932-a3e1-ae8a315534e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:36.039627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cd010-cd70-4932-a3e1-ae8a315534e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:36.041324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cd010-cd70-4932-a3e1-ae8a315534e9 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:36.053407 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1082/2170] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:36.628835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e614-192e-4b2b-bd72-52241403641a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:36.628835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3775e614-192e-4b2b-bd72-52241403641a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:36.654139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e614-192e-4b2b-bd72-52241403641a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:36.660076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:36.661049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:36.670923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:36.671284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] PATH_INFO: `/v3/projects/bfae378d88ba40cba43f703ec40d882b` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:36.672802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfae378d88ba40cba43f703ec40d882b)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:36.674761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:36.737094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfae378d88ba40cba43f703ec40d882b', 'request_id': 'req-a2353a4c-2d70-46e5-ad42-582a2786c09f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:36.751926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfae378d88ba40cba43f703ec40d882b', 'request_id': 'req-a2353a4c-2d70-46e5-ad42-582a2786c09f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:36.762299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfae378d88ba40cba43f703ec40d882b', 'request_id': 'req-a2353a4c-2d70-46e5-ad42-582a2786c09f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:36.780455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] The token cache is being invalidate because project bfae378d88ba40cba43f703ec40d882b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:54:36.780455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:36.780959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a2353a4c-2d70-46e5-ad42-582a2786c09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:54:36.783177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1089/2171] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:54:36 2022] DELETE /identity/v3/projects/bfae378d88ba40cba43f703ec40d882b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:54:36.791346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06c07-3aac-42dd-83f6-4828ffa7fc4f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:36.792519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06c07-3aac-42dd-83f6-4828ffa7fc4f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:36.793664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06c07-3aac-42dd-83f6-4828ffa7fc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:36.816705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ced06c07-3aac-42dd-83f6-4828ffa7fc4f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:36.818196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ced06c07-3aac-42dd-83f6-4828ffa7fc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:36.899518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1083/2172] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:36.906386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3881-c2d1-4441-8cb9-13d939f91c5b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:36.907001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3881-c2d1-4441-8cb9-13d939f91c5b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:36.907287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3881-c2d1-4441-8cb9-13d939f91c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:36.929766 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b02c3881-c2d1-4441-8cb9-13d939f91c5b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:36.930994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b02c3881-c2d1-4441-8cb9-13d939f91c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:36.944548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1090/2173] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:54:36.949029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06c07-3aac-42dd-83f6-4828ffa7fc4f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:36.957419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1064c-683a-4a77-addf-ca70d4eeee0f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:36.957753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1064c-683a-4a77-addf-ca70d4eeee0f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:36.958027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1064c-683a-4a77-addf-ca70d4eeee0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:36.970700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06c07-3aac-42dd-83f6-4828ffa7fc4f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:36.976734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:36.977443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:36.977727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:36.978006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:36.978761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:54:36.979070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:36.980767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:36.982569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-73d1064c-683a-4a77-addf-ca70d4eeee0f None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:36.984052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-73d1064c-683a-4a77-addf-ca70d4eeee0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:36.994051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:36.995410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1084/2174] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:54:36.999363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:37.010372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ca65c-bb74-4646-957b-62472786415f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:37.016827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:37.016827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.016827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.016827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.017360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:37.019322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:37.044569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1091/2175] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 12:54:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:54:37.046437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1085/2176] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:54:36 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:54:37.050852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1064c-683a-4a77-addf-ca70d4eeee0f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:37.059459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:37.061185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:37.062360 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1064c-683a-4a77-addf-ca70d4eeee0f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:37.070902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:37.123624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:37.124612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.124962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.125310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.126129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:37.128145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:37.144868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9f862-9dee-40c9-afdb-049ab441cb2f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:37.148464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1092/2177] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:54:37 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 12:54:37.152596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:37.153685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908323d-3bfd-4241-bf7f-6ff9c6086a3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:37.154469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908323d-3bfd-4241-bf7f-6ff9c6086a3e service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.154755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908323d-3bfd-4241-bf7f-6ff9c6086a3e service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.155033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908323d-3bfd-4241-bf7f-6ff9c6086a3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.163166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8153fc0-9ba4-4da2-b4e2-630aaa77ac33 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:37.167926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:37.168597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.168888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.169166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.169944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:37.171614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:37.181961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1093/2178] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:54:37.185880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:37.196985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad298183-d992-48c4-b693-d6b386c63147 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:37.201660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:37.202319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.202602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.202874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.203630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:37.205230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:37.216344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1094/2179] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 12:54:37.217349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908323d-3bfd-4241-bf7f-6ff9c6086a3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:37.219006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908323d-3bfd-4241-bf7f-6ff9c6086a3e service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:37.221333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:37.229372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1086/2180] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 12:54:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 12:54:37.231913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba505717-d9c5-4e9f-8d14-b1b930eba506 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:37.237409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:37.238305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.238585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.238855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] PATH_INFO: `/v3/projects/91f9f62d475d4663ae7c211c8adb78e8/users/31af58bc06a14e3ba8b709b7deb75341/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.255038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f9f62d475d4663ae7c211c8adb78e8, user_id=31af58bc06a14e3ba8b709b7deb75341, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:37.258976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:37.273991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1095/2181] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:54:37 2022] PUT /identity/v3/projects/91f9f62d475d4663ae7c211c8adb78e8/users/31af58bc06a14e3ba8b709b7deb75341/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:54:37.279200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7547-9bf1-4a82-9773-ee3aae4bfae3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.279599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7547-9bf1-4a82-9773-ee3aae4bfae3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.280669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7547-9bf1-4a82-9773-ee3aae4bfae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.308513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c12c7547-9bf1-4a82-9773-ee3aae4bfae3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:37.310344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c12c7547-9bf1-4a82-9773-ee3aae4bfae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:37.476546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1087/2182] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:54:37.486409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:37.488423 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:37.566681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48522c47-1ce8-4d3e-96c4-0723822aa0c3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:37.572133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:37.573063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:37.573456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:37.573739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:37.586304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:37.588251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:37.598887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1096/2183] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:38.554294 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2b316-b175-4b9d-b1ba-a182138047eb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:38.554856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2b316-b175-4b9d-b1ba-a182138047eb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:38.554972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2b316-b175-4b9d-b1ba-a182138047eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:38.585830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d7e2b316-b175-4b9d-b1ba-a182138047eb None None] MFA Rules not processed for user `31af58bc06a14e3ba8b709b7deb75341`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:38.586959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d7e2b316-b175-4b9d-b1ba-a182138047eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:38.640704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:38.669078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1088/2184] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:54:38.681547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:54:38.683187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:38.690109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:38.761003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be174c-79eb-43f6-93ae-cde8684df38d service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:54:38.764935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:54:38.765763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:38.766188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:38.766633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:38.783522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:54:38.785359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:54:38.810279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1097/2185] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:54:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:54:40.213123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd47b-d8b9-4071-b35f-a1f0e65c8c93 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:54:40.213752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd47b-d8b9-4071-b35f-a1f0e65c8c93 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:54:40.213752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd47b-d8b9-4071-b35f-a1f0e65c8c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:54:40.232740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-be8cd47b-d8b9-4071-b35f-a1f0e65c8c93 None None] MFA Rules not processed for user `31af58bc06a14e3ba8b709b7deb75341`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:54:40.233970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-be8cd47b-d8b9-4071-b35f-a1f0e65c8c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:54:40.241551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1089/2186] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:54:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:54:48.794422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=24 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:54:50.227008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=13 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:03.226288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:03.228462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:03.242558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099774b-b900-4cf7-896e-e42c91568e29 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:03.247825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:03.248722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:03.249008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:03.249285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] PATH_INFO: `/v3/users/1e2d8e227f804dcc90ad527c1f73795e` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:03.250506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e2d8e227f804dcc90ad527c1f73795e)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:03.252637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:03.299908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e2d8e227f804dcc90ad527c1f73795e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.316420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e2d8e227f804dcc90ad527c1f73795e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.325061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e2d8e227f804dcc90ad527c1f73795e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.332129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e2d8e227f804dcc90ad527c1f73795e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.332775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e2d8e227f804dcc90ad527c1f73795e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.338659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1098/2187] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:55:03 2022] DELETE /identity/v3/users/1e2d8e227f804dcc90ad527c1f73795e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:55:03.343133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:03.343634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:03.343759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:03.362829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:55:03.364064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:03.422966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1090/2188] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:55:03.433268 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:03.434871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:03.498980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ace040-41ab-4d22-b96a-ef799b652d8f admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:03.502908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c4ec0-2b21-4382-b649-2c2153ffadad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:03.503596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c4ec0-2b21-4382-b649-2c2153ffadad service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:03.503883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c4ec0-2b21-4382-b649-2c2153ffadad service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:03.504163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c4ec0-2b21-4382-b649-2c2153ffadad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:03.513540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c4ec0-2b21-4382-b649-2c2153ffadad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:03.514983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c4ec0-2b21-4382-b649-2c2153ffadad service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:03.525080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1099/2189] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:55:03.834670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:03.836361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:03.849185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa223b-14f0-4418-8a90-7498db05683c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:03.854346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:03.855241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:03.855553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:03.855825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] PATH_INFO: `/v3/projects/52d0fdeac3734a478ae6308ba21ba6d8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:03.857113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d0fdeac3734a478ae6308ba21ba6d8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:03.859055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:03.918818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d0fdeac3734a478ae6308ba21ba6d8', 'request_id': 'req-1e6ca3bf-b882-4443-9958-61a09f3bd30b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.925187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d0fdeac3734a478ae6308ba21ba6d8', 'request_id': 'req-1e6ca3bf-b882-4443-9958-61a09f3bd30b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.931512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d0fdeac3734a478ae6308ba21ba6d8', 'request_id': 'req-1e6ca3bf-b882-4443-9958-61a09f3bd30b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.945824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] The token cache is being invalidate because project 52d0fdeac3734a478ae6308ba21ba6d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:55:03.946117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.946753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1e6ca3bf-b882-4443-9958-61a09f3bd30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:03.948610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1091/2190] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:55:03 2022] DELETE /identity/v3/projects/52d0fdeac3734a478ae6308ba21ba6d8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:55:03.953487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb93af-56fa-4215-8794-b29edf9a718b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:03.953782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb93af-56fa-4215-8794-b29edf9a718b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:03.954062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb93af-56fa-4215-8794-b29edf9a718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:03.972109 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-47bb93af-56fa-4215-8794-b29edf9a718b None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:55:03.973341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-47bb93af-56fa-4215-8794-b29edf9a718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:04.045184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1100/2191] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:55:04.054076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc630-4281-40cd-b07c-030eae2a7a16 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.054384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc630-4281-40cd-b07c-030eae2a7a16 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.054666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc630-4281-40cd-b07c-030eae2a7a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.078844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ecfbc630-4281-40cd-b07c-030eae2a7a16 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:55:04.079854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ecfbc630-4281-40cd-b07c-030eae2a7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:04.094403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1092/2192] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:55:04.098767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb93af-56fa-4215-8794-b29edf9a718b None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:04.108889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb93af-56fa-4215-8794-b29edf9a718b None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:04.115426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:04.120861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.120861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.120861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.120861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 12:55:04.120861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:04.120861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:04.134029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1101/2193] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 12:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 12:55:04.138545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbc630-4281-40cd-b07c-030eae2a7a16 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:04.150428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbc630-4281-40cd-b07c-030eae2a7a16 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:04.156426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:04.157253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.157515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.157791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.158584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:04.160935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:04.200851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1093/2194] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 12:55:04 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:55:04.202132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:04.208770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:04.214411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9777c-04b7-49d4-85fc-88d55b47475d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:04.278883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:04.279628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.279877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.280156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.280855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:04.282800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:04.307985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1102/2195] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 12:55:04 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 12:55:04.310864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:04.324779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c41839-bf74-4802-a0ba-538c84bc66ee admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:04.329904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:04.330678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.330963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.331239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.332033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:04.333652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:04.346841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1094/2196] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:55:04.351704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:04.364210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b4b0c-2b23-4385-b8c1-611d05e548ff admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:04.374265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22e9c2-3cd1-4cfa-b1e8-756997454b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:04.374924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e9c2-3cd1-4cfa-b1e8-756997454b41 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.375182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e9c2-3cd1-4cfa-b1e8-756997454b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.375557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e9c2-3cd1-4cfa-b1e8-756997454b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.376989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22e9c2-3cd1-4cfa-b1e8-756997454b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:04.378076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22e9c2-3cd1-4cfa-b1e8-756997454b41 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:04.398937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1103/2197] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 12:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 12:55:04.403106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:04.414607 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996faa3-7ed7-49a4-beb9-db128933a5ef admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:04.419583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:04.420549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.420839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.421291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] PATH_INFO: `/v3/projects/0ec3b2bdd8d8486bafccfb68012f378f/users/5667988d73634e3db162d81ed95ffc56/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.442288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec3b2bdd8d8486bafccfb68012f378f, user_id=5667988d73634e3db162d81ed95ffc56, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:04.454980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:04.473536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1095/2198] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 12:55:04 2022] PUT /identity/v3/projects/0ec3b2bdd8d8486bafccfb68012f378f/users/5667988d73634e3db162d81ed95ffc56/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 12:55:04.485558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70231c-2dab-4e30-a5f2-a2124ad0b9f3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.486056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70231c-2dab-4e30-a5f2-a2124ad0b9f3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.486421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70231c-2dab-4e30-a5f2-a2124ad0b9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.525393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-5f70231c-2dab-4e30-a5f2-a2124ad0b9f3 None None] MFA Rules not processed for user `5667988d73634e3db162d81ed95ffc56`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:55:04.527008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5f70231c-2dab-4e30-a5f2-a2124ad0b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:04.609744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1104/2199] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:55:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 12:55:04.621911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:04.623942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:04.652317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:04.720466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741de6e4-e5e6-43d6-b93b-4de272f61f37 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:04.724523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:04.725183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:04.725470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:04.725745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:04.745072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:04.746557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:04.756940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1096/2200] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:55:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:55:14.609940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:14.751831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=26 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:27.670544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb0105-6b4e-4aeb-bcb3-77812d466f2c None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:27.671031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb0105-6b4e-4aeb-bcb3-77812d466f2c None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:27.671031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb0105-6b4e-4aeb-bcb3-77812d466f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:27.727093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4ccb0105-6b4e-4aeb-bcb3-77812d466f2c None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:55:27.728527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4ccb0105-6b4e-4aeb-bcb3-77812d466f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:27.797574 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1105/2201] 10.222.0.87 () {58 vars in 967 bytes} [Fri Jun 17 12:55:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:55:27.822731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:27.824423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:27.929826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67230c-4d8f-41c3-9868-e314b5548c1b service glance] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:27.936216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3bc9c-bb02-4a92-9080-72f67c36420e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:27.936883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bc9c-bb02-4a92-9080-72f67c36420e service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:27.937171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bc9c-bb02-4a92-9080-72f67c36420e service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:27.937450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bc9c-bb02-4a92-9080-72f67c36420e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:27.947256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3bc9c-bb02-4a92-9080-72f67c36420e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:27.948906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3bc9c-bb02-4a92-9080-72f67c36420e service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:27.957389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1097/2202] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 12:55:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 12:55:37.794132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=40 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:37.953993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=41 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:48.546417 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb0105-6b4e-4aeb-bcb3-77812d466f2c None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:48.620526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb0105-6b4e-4aeb-bcb3-77812d466f2c None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:48.625795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:48.626667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:48.626944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:48.627212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] PATH_INFO: `/v3/users/5667988d73634e3db162d81ed95ffc56` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:48.628480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5667988d73634e3db162d81ed95ffc56)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:48.630415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:48.688375 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5667988d73634e3db162d81ed95ffc56'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:48.701774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5667988d73634e3db162d81ed95ffc56'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:48.711580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5667988d73634e3db162d81ed95ffc56'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:48.718575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5667988d73634e3db162d81ed95ffc56'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:48.719229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5667988d73634e3db162d81ed95ffc56'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:48.725756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1106/2203] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 12:55:48 2022] DELETE /identity/v3/users/5667988d73634e3db162d81ed95ffc56 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:55:48.730903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:48.731434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:48.731545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:48.749350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:55:48.750591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:48.810015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1098/2204] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:55:48.820005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:48.821518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:48.882369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8895f-b22c-4f37-9ae1-de332865d759 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:48.886876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:48.887554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:48.887849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:48.888104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:48.897964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:48.899426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:48.909119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1107/2205] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:55:49.206539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:55:49.208661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:55:49.226398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215f6f4-565c-49ce-bb06-bc8f76d720f5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:55:49.237401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:55:49.238427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:55:49.238717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:55:49.238994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] PATH_INFO: `/v3/projects/0ec3b2bdd8d8486bafccfb68012f378f` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:55:49.240927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec3b2bdd8d8486bafccfb68012f378f)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:55:49.243043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:55:49.294630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec3b2bdd8d8486bafccfb68012f378f', 'request_id': 'req-fd1cfddc-d60a-4cd5-81e9-40b293147878'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:49.300582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec3b2bdd8d8486bafccfb68012f378f', 'request_id': 'req-fd1cfddc-d60a-4cd5-81e9-40b293147878'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:49.306213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec3b2bdd8d8486bafccfb68012f378f', 'request_id': 'req-fd1cfddc-d60a-4cd5-81e9-40b293147878'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:49.320704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] The token cache is being invalidate because project 0ec3b2bdd8d8486bafccfb68012f378f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:55:49.321104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:49.321704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:55:49.323682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1099/2206] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 12:55:49 2022] DELETE /identity/v3/projects/0ec3b2bdd8d8486bafccfb68012f378f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:55:58.904738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:55:59.317891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:56:40.477319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:56:40.478986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:40.555114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838d9b5-768e-4a20-9baa-99a11443f6d0 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:56:40.632152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:56:40.632802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:56:40.633091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:56:40.633353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:56:40.635282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:40.693323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:56:40.694987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:56:40.704290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1108/2207] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:56:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:56:44.017678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:56:44.020439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:44.038519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cfddc-d60a-4cd5-81e9-40b293147878 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:56:44.045214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:56:44.045880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:56:44.046159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:56:44.046430 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:56:44.048524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:44.109070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:56:44.110712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:56:44.120656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1100/2208] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:56:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:56:44.314334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:56:44.317095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:44.360217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:56:44.377428 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd2724-843a-4a7a-a135-3a126125e7d6 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:56:44.381344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:56:44.381998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:56:44.382273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:56:44.382547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:56:44.384433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:44.443076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:56:44.444585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:56:44.453767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1109/2209] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:56:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:56:47.547134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:56:47.551719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:47.637160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cd8c4-2ff9-4d99-8f97-bfa6308c57e8 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:56:47.640910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:56:47.641572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:56:47.641861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:56:47.642141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:56:47.644005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:47.658948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:56:47.660445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:56:47.672133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1101/2210] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:56:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:56:48.864864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:56:48.875438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0fee5-2998-4ed8-8a01-50aa0753c7c6 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:56:48.879157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:56:48.879847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:56:48.880130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:56:48.880405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:56:48.882196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:48.896154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:56:48.897629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:56:48.911814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1110/2211] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:56:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 12:56:50.676896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:56:50.687139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454c4f6-2939-4336-b614-33a6006af069 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:56:50.691732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd1cd1-44ad-43fa-af8e-7895ca5bd058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:56:50.692389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd1cd1-44ad-43fa-af8e-7895ca5bd058 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:56:50.692673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd1cd1-44ad-43fa-af8e-7895ca5bd058 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:56:50.692952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd1cd1-44ad-43fa-af8e-7895ca5bd058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:56:50.694945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d2dd1cd1-44ad-43fa-af8e-7895ca5bd058 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:50.758146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd1cd1-44ad-43fa-af8e-7895ca5bd058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:56:50.759652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd1cd1-44ad-43fa-af8e-7895ca5bd058 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:56:50.769512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1102/2212] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:56:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:56:57.795745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:56:57.797432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:57.809916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0a56a-b2b9-44be-9cde-bb2e90c68d77 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:56:57.814498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:56:57.815177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:56:57.815503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:56:57.815782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:56:57.817750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:56:57.827755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:56:57.829168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:56:57.837251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1111/2213] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:56:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:57:00.764587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=82 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:57:03.542564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:57:03.543440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:57:03.543440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:57:03.562262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] MFA Rules not processed for user `efee42b434cd47d99984b85f677983c2`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 12:57:03.563518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:57:03.572538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1103/2214] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 12:57:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 12:57:07.833767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=44 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:57:12.469844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:57:12.472297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:57:12.490470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bf271-c0fc-44b4-83bb-179a1ae9a1dc service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:57:12.496758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:57:12.497496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:57:12.497787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:57:12.498066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:57:12.500067 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:57:12.516755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:57:12.518376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:57:12.530535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1112/2215] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:57:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:57:13.557057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:57:22.526074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:58:48.905455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:58:48.907081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:58:48.919892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:58:48.922050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a1543-e30e-487f-8eb2-24f9769f735b None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:58:48.926202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:58:48.926876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:58:48.927165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:58:48.927478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:58:48.929362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:58:48.932341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152952d6-4966-417d-957d-7d8d5c8f3105 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:58:48.938373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:58:48.939070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:58:48.939425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:58:48.939794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:58:48.941964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:58:48.992958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:58:48.994457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:58:49.003686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1104/2216] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:58:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:58:49.006023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:58:49.007661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:58:49.017026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1113/2217] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:58:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:58:58.999577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=28 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:58:59.012698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=28 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:59:05.258410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:59:05.273752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14faa-9988-407f-9654-51aca5ea18c2 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:59:05.280255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:59:05.280914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:59:05.281202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:59:05.281506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:59:05.283501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:59:05.297950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:59:05.299604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:59:05.311048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1105/2218] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:59:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:59:15.307305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=6 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:59:39.940444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:59:39.942309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:59:40.015996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf027b71-e28c-4526-adc9-d097549433a7 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:59:40.021866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:59:40.022550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:59:40.022830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:59:40.023093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:59:40.025065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:59:40.038176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:59:40.039822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:59:40.050014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1114/2219] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 12:59:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:59:50.045620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=32 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:01:00.399295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:01:00.413280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478943a8-721c-47b4-b690-c885cabfacbd service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:01:00.417348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:01:00.418024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:01:00.418288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:01:00.418566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:01:00.420579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:01:00.490053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:01:00.491569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:01:00.503403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1106/2220] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:01:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:01:10.497253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=28 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:01:40.040850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:01:40.053669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d42a4-a429-421e-9ed7-a446a3bb5026 service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:01:40.059916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:01:40.060572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:01:40.060858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:01:40.061133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:01:40.069357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:01:40.070913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:01:40.099284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1115/2221] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:01:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:01:43.637559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:01:43.650784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f82a2-1680-4c18-9a05-b414cdae5b2d service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:01:43.657693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:01:43.658341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:01:43.658627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:01:43.658899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:01:43.667187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:01:43.668839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:01:43.698731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1107/2222] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:01:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:01:47.026722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:01:47.036324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ce1ef-d349-424d-9eb0-c1ff7c45e857 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:01:47.040291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:01:47.041162 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:01:47.041449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:01:47.041724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:01:47.052552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:01:47.054022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:01:47.065704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1116/2223] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:01:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:01:53.695684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=13 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:01:57.060902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=19 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:02:00.773268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:02:00.794191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e661472-a56c-4a1f-ab53-952a3cbfd154 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:02:00.804854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:02:00.805799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:02:00.806213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:02:00.806570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:02:00.820662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:02:00.822270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:02:00.851915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1108/2224] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:02:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:02:10.848885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=13 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:02:42.566497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:02:42.575934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2f1e4-044c-4ad8-824a-a7c54d932655 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:02:42.580910 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:02:42.581563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:02:42.581852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:02:42.582121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:02:42.589959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:02:42.591404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:02:42.600119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1117/2225] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:02:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:02:52.596767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:03:00.729601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:03:00.743223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aee5a0-df14-4631-8d4d-92d588b39dad service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:03:00.747852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:03:00.748510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:03:00.748792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:03:00.749064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:03:00.761378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:03:00.762841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:03:00.774275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1109/2226] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:03:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:03:00.830381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:03:00.841053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d305d4-87c5-4dcd-90b8-205d03e101c8 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:03:00.850536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:03:00.851190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:03:00.851495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:03:00.851769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:03:00.862770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:03:00.864282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:03:00.894778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1118/2227] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:03:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:03:10.769219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=6 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:03:10.892022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=13 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:04:45.824425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:04:45.825964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:04:45.924961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29c91-7f82-454b-a79c-5606dffe2bb1 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:04:45.931423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:04:45.932152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:04:45.932443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:04:45.932718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:04:45.943235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:04:45.944931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:04:45.956636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1110/2228] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:04:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:04:49.123425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:04:49.158515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14aaaf-168e-41cb-9ce1-5fa8a3095fe9 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:04:49.162602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:04:49.163248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:04:49.163564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:04:49.163843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:04:49.172814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:04:49.174256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:04:49.203408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1119/2229] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:04:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:04:55.951404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=37 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:04:59.200524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=22 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:05:01.241641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:05:01.271952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37e2ac-ec70-4b07-ba2a-5c28b7741a7e service placement] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:05:01.277999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:05:01.278693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:05:01.278997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:05:01.279276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:05:01.287600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:05:01.289238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:05:01.317982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1111/2230] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:05:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:05:11.315252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=20 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:07:01.347799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:07:01.349570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:07:01.424716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ff1a-4e1c-4f30-b437-752d8304d7a1 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:07:01.428468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:07:01.429130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:07:01.429416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:07:01.429693 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:07:01.431699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:07:01.522406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:07:01.523981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:07:01.537256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1120/2231] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:07:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:07:01.544933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:07:01.555395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e564e-97e8-4d29-96bf-b4eb4d5f51d6 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:07:01.559411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:07:01.560058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:07:01.560341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:07:01.560616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:07:01.563426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:07:01.642632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:07:01.644127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:07:01.653879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1112/2232] 10.222.0.87 () {60 vars in 1396 bytes} [Fri Jun 17 13:07:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:07:01.712253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:07:01.720887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d008a-2fe5-4cad-90cc-709b9bbe5604 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:07:01.724870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:07:01.725524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:07:01.725804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:07:01.726077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:07:01.735914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:07:01.737353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:07:01.765427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1121/2233] 10.222.0.87 () {60 vars in 1396 bytes} [Fri Jun 17 13:07:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:07:11.649699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=33 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:07:11.762175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=77 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:07:47.487777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:07:47.489542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:07:47.576576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9314b15-b95b-433a-a28a-ddb31f332128 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:07:47.580754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:07:47.581414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:07:47.581702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:07:47.581953 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:07:47.583955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:07:47.593846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:07:47.595289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:07:47.603434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1113/2234] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:07:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:07:57.599901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=35 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:08:49.273107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:08:49.274796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:08:49.287843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcbd11-7ed4-43a5-95a4-999eeb71eb2a service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:08:49.292464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:08:49.293140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:08:49.293460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:08:49.293732 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:08:49.295907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:08:49.370852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:08:49.372387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:08:49.381650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1122/2235] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:08:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:08:59.377812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=33 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:09:01.913641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:09:01.923767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf1ee1-8f8f-4ba4-b693-e8c6263c01ed service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:09:01.928626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:09:01.929266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:09:01.929542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:09:01.929813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:09:01.932002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:09:02.008885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:09:02.010347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:09:02.020632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1114/2236] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:09:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:09:02.170475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:09:02.179682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882993-77c3-4016-ae47-6bf5eb692670 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:09:02.183775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:09:02.184470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:09:02.184889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:09:02.185057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:09:02.187326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:09:02.202504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:09:02.204113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:09:02.217443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1123/2237] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:09:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:09:02.395347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:09:02.396994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:09:02.413134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76626a2f-c500-44ed-8333-92d8a3ae6920 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:09:02.419072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:09:02.419748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:09:02.420036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:09:02.420314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:09:02.422179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:09:02.493920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:09:02.495652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:09:02.505050 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1115/2238] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:09:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:09:12.212253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:09:12.500857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=66 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:09:51.777644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:09:51.779217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:09:51.851220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22500f-02a7-4f4d-b356-5a5486d79281 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:09:51.856707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:09:51.857362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:09:51.857643 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:09:51.857917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:09:51.859907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:09:51.873529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:09:51.875125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:09:51.885690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1124/2239] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:09:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:10:01.881137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=32 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:10:49.468724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:10:49.479348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4de4-bc59-4990-99a5-951d25304257 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:10:49.484089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:10:49.484758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:10:49.485046 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:10:49.485328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:10:49.487735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:10:49.566306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:10:49.567304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:10:49.577138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1116/2240] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:10:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:10:59.573743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=33 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:12:49.664274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:49.674237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad6318-18a7-4360-a2ac-6b23265a05e3 service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:49.678351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:49.679057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:49.679348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:49.679633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:49.681685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:49.760868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:49.762318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:49.771657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1125/2241] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:12:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:12:49.823137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:49.832465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614d6f9-5514-48ff-89fb-e7651145fe6e service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:49.836679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:49.837330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:49.837595 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:49.837899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:49.849587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:49.851080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:49.862133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1117/2242] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:12:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:12:49.869089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:49.878880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd57e-f691-4e12-ac2e-04d9ccb225c3 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:49.883875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:49.884514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:49.884792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:49.885064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:49.894401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:49.895862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:49.924372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1126/2243] 10.222.0.87 () {60 vars in 1396 bytes} [Fri Jun 17 13:12:49 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:12:50.044287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:50.051239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc541c5-d697-41d8-915f-31ddb9742f96 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:50.058506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f59a0-2a13-47f3-8d7d-3f207cd4d4f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:50.059509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f59a0-2a13-47f3-8d7d-3f207cd4d4f3 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:50.059828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f59a0-2a13-47f3-8d7d-3f207cd4d4f3 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:50.060080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f59a0-2a13-47f3-8d7d-3f207cd4d4f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:50.066348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f14f59a0-2a13-47f3-8d7d-3f207cd4d4f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:50.160317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f59a0-2a13-47f3-8d7d-3f207cd4d4f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:50.161969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f59a0-2a13-47f3-8d7d-3f207cd4d4f3 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:50.171282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1118/2244] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:12:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:12:52.459393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:52.470111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485c9e-f3d1-4992-ab9a-a62b2bf18b79 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:52.472660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:52.473204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:52.473487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:52.473765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:52.481785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:52.483640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:52.492419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1127/2245] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:12:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:12:52.767692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:52.768199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:52.768383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:52.795822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] MFA Rules not processed for user `0549a0df5e204b2b97bdcf4c745042c7`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:12:52.797091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:52.804506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1119/2246] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:12:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:12:53.147935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:53.149572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:53.161333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759f6c0-923c-448e-a352-74deaff9fc2a service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:53.165179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:53.165813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:53.166110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:53.166381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:53.168197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:53.178307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:53.179893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:53.188553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1128/2247] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:12:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:12:54.292811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:54.294519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:54.311775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470462-d2f3-4dc3-afcd-8c66a4e6b624 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:54.318015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:54.318689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:54.318976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:54.319264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:54.321439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:54.334840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:54.336752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:54.348344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1120/2248] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:12:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:12:55.479716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:12:55.494431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d013e-6278-49b5-81cd-6c6ddffb635f service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:12:55.501136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:12:55.501901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:12:55.502247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:12:55.502585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:12:55.504715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:12:55.518647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:12:55.520274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:12:55.532304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1129/2249] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:12:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:13:04.344382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=51 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:13:05.526955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=64 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:14:01.632302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:14:01.645320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d97a-7d2d-4839-bb7e-e14be5b33115 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:14:01.652621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:14:01.653276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:14:01.653562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:14:01.653846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:14:01.662567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:14:01.664227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:14:01.692417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1121/2250] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:14:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:14:02.204705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:14:02.214987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da8778-ccd6-417d-8fff-0413f5ebd76e service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:14:02.219759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:14:02.220408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:14:02.220696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:14:02.220976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:14:02.232185 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:14:02.233720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:14:02.245133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1130/2251] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:14:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:14:11.689930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=13 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:14:12.240164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:14:55.818912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:14:55.871181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c214c6-5f8c-4398-a4e1-8eebe8734f03 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:14:55.876077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:14:55.877159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:14:55.877451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:14:55.877741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:14:55.903825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:14:55.905299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:14:55.933997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1122/2252] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:14:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:14:55.991763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:14:56.014802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853835e-bfb1-4c12-aad8-411c697a50d9 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:14:56.018828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:14:56.019510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:14:56.019796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:14:56.020075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:14:56.029171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:14:56.030615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:14:56.059710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1131/2253] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:14:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:14:57.755401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:14:57.791655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939ec5-99e3-4574-9c8d-95a28996c442 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:14:57.797842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:14:57.798504 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:14:57.798828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:14:57.799213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:14:57.808813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:14:57.811353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:14:57.822444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1123/2254] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:14:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:15:06.056614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=18 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:15:07.817947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=44 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:16:56.493451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:16:56.503460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a552-cbe5-46b3-9612-8e8bc540a765 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:16:56.507778 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:16:56.508520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:16:56.508855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:16:56.509200 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:16:56.518566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:16:56.520202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:16:56.549033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1132/2255] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:16:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:17:06.546218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=13 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:17:50.778539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:17:50.883025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4dd24-2e25-4394-8929-96a4d6dc2751 service placement] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:17:50.889559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:17:50.890207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:17:50.890499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:17:50.890787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:17:50.892898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:17:50.971796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:17:50.975156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:17:50.976819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:17:50.985030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397f869-e2c3-4b97-8fd6-79479e17eba9 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:17:50.986259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1124/2256] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:17:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:17:50.991107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:17:50.991829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:17:50.992458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:17:50.992458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:17:51.001620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:17:51.003210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:17:51.031996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1133/2257] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:17:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:17:51.084913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:17:51.162846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfcae3-2182-474e-ac7c-a4e84cfc37fa service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:17:51.167061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:17:51.167808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:17:51.168079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:17:51.168354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:17:51.170490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:17:51.175291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:17:51.202035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:17:51.203563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:17:51.215202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1125/2258] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:17:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:17:58.436196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:17:58.448633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f89c-6078-4fff-970c-ab60ba2aff91 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:17:58.453863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:17:58.454511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:17:58.454786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:17:58.455086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:17:58.457274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:17:58.482316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:17:58.483847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:17:58.495238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1134/2259] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:18:01.209727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=7 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:18:08.488795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=24 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:18:54.868986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:18:54.870601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:18:54.881642 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ec539-bf32-4d9c-a158-e1691d38c367 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:18:54.885601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:18:54.886260 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:18:54.886543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:18:54.886816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:18:54.888686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:18:54.978226 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:18:54.979612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:18:54.999300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1126/2260] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:18:56.819794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:18:56.822014 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:18:56.901605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9126-2c08-4421-822e-eda306f6cb46 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:18:56.905808 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:18:56.906436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:18:56.906715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:18:56.906988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:18:56.917613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:18:56.919069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:18:56.950626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1135/2261] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:18:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:18:58.066470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:18:58.076832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1ebd7-ca54-4b38-9f45-e67a73b110d6 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:18:58.080501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:18:58.081157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:18:58.081435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:18:58.081732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:18:58.083539 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:18:58.097748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:18:58.099475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:18:58.110802 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1127/2262] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:18:59.338273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:18:59.352448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16627b-421e-467c-9a56-1efb2657bf7b service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:18:59.359474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:18:59.360333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:18:59.360786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:18:59.361225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:18:59.363523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:18:59.400973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:18:59.403565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:18:59.418564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1136/2263] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:18:59.609847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:18:59.628341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02585033-9ce3-4a12-86ec-94312e52485f service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:18:59.648019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:18:59.649314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:18:59.650364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:18:59.652922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:18:59.676544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:18:59.679237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:18:59.711426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1128/2264] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:19:03.003854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:03.016388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d6282-0745-47c2-9a36-81f22ec16ca3 service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:03.021086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:03.022210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:03.032105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:03.032538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:03.034934 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:03.049386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:03.050843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:03.062980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1137/2265] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:19:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:19:03.221582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:03.223943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:03.238194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c10ed-d85a-415d-bb77-0dd3f083e2ea service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:03.243182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:03.244563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:03.244853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:03.245131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:03.247096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:03.375570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:03.377119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:03.391452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1129/2266] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:19:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:19:04.780322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:04.782145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:04.797562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034169-c7e4-45df-8915-6fcff604d50a service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:04.803955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:04.804811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:04.805116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:04.805401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:04.808068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:04.820680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:04.822355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:04.836326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1138/2267] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:19:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:19:04.868703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:04.870896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:04.889280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e1a0d-99ef-4bd3-a420-73087c5a57f0 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:04.894760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:04.895692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:04.895975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:04.896247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] PATH_INFO: `/v3/users/109a46a7108e48c48407681b3e5b1e0c` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:04.898153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109a46a7108e48c48407681b3e5b1e0c)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:04.900282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:04.923559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:04.951512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109a46a7108e48c48407681b3e5b1e0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:04.963684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109a46a7108e48c48407681b3e5b1e0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:04.970597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109a46a7108e48c48407681b3e5b1e0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:04.980411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109a46a7108e48c48407681b3e5b1e0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:04.981152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109a46a7108e48c48407681b3e5b1e0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:04.987398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1130/2268] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:19:04 2022] DELETE /identity/v3/users/109a46a7108e48c48407681b3e5b1e0c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:05.181700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:05.193449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355afea-6812-4616-aeef-d0a516590138 service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:05.270642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:05.271647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:05.271960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:05.272234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] PATH_INFO: `/v3/projects/3b5f6238effd43d39cbf0af0f4c582f8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:05.273507 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b5f6238effd43d39cbf0af0f4c582f8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:05.275575 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:05.311884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:05.333762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b5f6238effd43d39cbf0af0f4c582f8', 'request_id': 'req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.341790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b5f6238effd43d39cbf0af0f4c582f8', 'request_id': 'req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.350309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b5f6238effd43d39cbf0af0f4c582f8', 'request_id': 'req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.370701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] The token cache is being invalidate because project 3b5f6238effd43d39cbf0af0f4c582f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:19:05.371150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.372639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.375251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1139/2269] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:19:05 2022] DELETE /identity/v3/projects/3b5f6238effd43d39cbf0af0f4c582f8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:05.379027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:05.380482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:05.394461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b11e7-7cdf-43f7-8510-e5b6c20bd714 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:05.471463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:05.473673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:05.474693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:05.475338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] PATH_INFO: `/v3/users/63c51101d06444218eb60b017ee40f78` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:05.494715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c51101d06444218eb60b017ee40f78)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:05.496917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:05.573186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c51101d06444218eb60b017ee40f78'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.593039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c51101d06444218eb60b017ee40f78'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.602559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c51101d06444218eb60b017ee40f78'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.615297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c51101d06444218eb60b017ee40f78'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.616620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c51101d06444218eb60b017ee40f78'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:05.633741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1131/2270] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:19:05 2022] DELETE /identity/v3/users/63c51101d06444218eb60b017ee40f78 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:05.845026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:05.846846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:05.888577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dbfd2-e3c2-45ca-a354-c487a0271c35 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:05.967605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:05.968618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:05.968970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:05.969403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] PATH_INFO: `/v3/projects/df692fbae8174caf80369f4e4f5832c2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:05.980772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df692fbae8174caf80369f4e4f5832c2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:05.982827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:06.032980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df692fbae8174caf80369f4e4f5832c2', 'request_id': 'req-18aa4eed-5fd0-4123-80a3-49d8915111d7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:06.040377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df692fbae8174caf80369f4e4f5832c2', 'request_id': 'req-18aa4eed-5fd0-4123-80a3-49d8915111d7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:06.049566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df692fbae8174caf80369f4e4f5832c2', 'request_id': 'req-18aa4eed-5fd0-4123-80a3-49d8915111d7'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:06.067823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] The token cache is being invalidate because project df692fbae8174caf80369f4e4f5832c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:19:06.068196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:06.069072 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:06.071048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1140/2271] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:19:05 2022] DELETE /identity/v3/projects/df692fbae8174caf80369f4e4f5832c2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:07.105605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:07.107164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:07.125114 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d895d21-caa4-458f-b8cb-816dd6e60bb3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:07.129179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:07.184428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:07.185322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:07.185594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:07.185870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] PATH_INFO: `/v3/users/0549a0df5e204b2b97bdcf4c745042c7` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:07.187167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0549a0df5e204b2b97bdcf4c745042c7)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:07.189381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:07.236364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0549a0df5e204b2b97bdcf4c745042c7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.251831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0549a0df5e204b2b97bdcf4c745042c7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.259345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0549a0df5e204b2b97bdcf4c745042c7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.267903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0549a0df5e204b2b97bdcf4c745042c7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.268569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0549a0df5e204b2b97bdcf4c745042c7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.276981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1132/2272] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:19:07 2022] DELETE /identity/v3/users/0549a0df5e204b2b97bdcf4c745042c7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:07.507913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:07.509437 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:07.522860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa4eed-5fd0-4123-80a3-49d8915111d7 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:07.560257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:07.585207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:07.586269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:07.586578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:07.586858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] PATH_INFO: `/v3/projects/6eae9ef94ebe47e7a585d24d25930a82` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:07.588103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eae9ef94ebe47e7a585d24d25930a82)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:07.590028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:07.642623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eae9ef94ebe47e7a585d24d25930a82', 'request_id': 'req-535222b5-1781-4404-aad3-c416e6b07abb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.655184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eae9ef94ebe47e7a585d24d25930a82', 'request_id': 'req-535222b5-1781-4404-aad3-c416e6b07abb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.662612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eae9ef94ebe47e7a585d24d25930a82', 'request_id': 'req-535222b5-1781-4404-aad3-c416e6b07abb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.676514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] The token cache is being invalidate because project 6eae9ef94ebe47e7a585d24d25930a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:19:07.676802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.677690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.679740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1141/2273] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:19:07 2022] DELETE /identity/v3/projects/6eae9ef94ebe47e7a585d24d25930a82 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:07.683565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:07.684969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:07.697688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07ec44-ed00-484d-9972-29c9ca79b3ec admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:07.754765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:07.755693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:07.755977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:07.756271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] PATH_INFO: `/v3/users/d89a797e40164eff97d1696cab44be6e` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:07.769722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89a797e40164eff97d1696cab44be6e)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:07.771701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:07.786597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:07.826857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89a797e40164eff97d1696cab44be6e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.842319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89a797e40164eff97d1696cab44be6e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.849940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89a797e40164eff97d1696cab44be6e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.859562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89a797e40164eff97d1696cab44be6e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.860325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89a797e40164eff97d1696cab44be6e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:07.868601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1133/2274] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:19:07 2022] DELETE /identity/v3/users/d89a797e40164eff97d1696cab44be6e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:08.078210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:08.080865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:08.102642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535222b5-1781-4404-aad3-c416e6b07abb admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:08.156045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:08.156925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:08.157209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:08.157481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] PATH_INFO: `/v3/projects/372932232b17459b8f39b39a9b55100a` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:08.168828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] RBAC: Authorizing `identity:delete_project(project_id=372932232b17459b8f39b39a9b55100a)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:08.170764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:08.223023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372932232b17459b8f39b39a9b55100a', 'request_id': 'req-936f205a-34ef-4ecf-985e-1fc229a8958f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:08.230009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372932232b17459b8f39b39a9b55100a', 'request_id': 'req-936f205a-34ef-4ecf-985e-1fc229a8958f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:08.234607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:08.236017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372932232b17459b8f39b39a9b55100a', 'request_id': 'req-936f205a-34ef-4ecf-985e-1fc229a8958f'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:08.251590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] The token cache is being invalidate because project 372932232b17459b8f39b39a9b55100a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:19:08.251920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:08.252581 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:19:08.254291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1142/2275] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:19:08 2022] DELETE /identity/v3/projects/372932232b17459b8f39b39a9b55100a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:19:17.863700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=9,keystone:SELECT=20,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:18.248497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=3,keystone:INSERT=1,keystone:DELETE=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:19:55.912081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:55.913781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:55.995754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76879c1-7ecc-4351-a4dd-64f843a7b149 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:56.064448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:56.065129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:56.065406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:56.065730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:56.067946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:56.142659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:56.144319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:56.154093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1134/2276] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:19:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:19:56.367591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:19:56.369258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:56.387263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f205a-34ef-4ecf-985e-1fc229a8958f admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:19:56.394032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:19:56.394729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:19:56.395012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:19:56.395288 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:19:56.397580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:19:56.476730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:19:56.478271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:19:56.488405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1143/2277] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:19:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:20:01.656804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:20:01.658531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:20:01.687338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:20:01.735795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40943-6475-4e0e-a980-f584f68338f4 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:20:01.742272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:20:01.743015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:20:01.743316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:20:01.743608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:20:01.745905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:20:01.758072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:20:01.759739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:20:01.769980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1135/2278] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:20:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:20:06.483896 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=33 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:20:11.765599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=21 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:22:51.037627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:22:51.039283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:22:51.102151 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf97028-559c-47da-81ad-0e8d77366480 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:22:51.106926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:22:51.107886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:22:51.108272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:22:51.108664 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:22:51.111587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:22:51.129179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:22:51.130645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:22:51.142466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1144/2279] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:22:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:23:01.137466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=26 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:23:03.418988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:23:03.429435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ab60c-909c-45f4-8c6e-e56ff5d7ccfe service placement] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:23:03.434424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:23:03.435100 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:23:03.435513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:23:03.435796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:23:03.438048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:23:03.450348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:23:03.451853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:23:03.462519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1136/2280] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:23:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:23:03.706825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:23:03.708558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:23:03.779651 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6796709-cd85-46ee-a40b-0c207c563d1f service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:23:03.783341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:23:03.784049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:23:03.784334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:23:03.784605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:23:03.786489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:23:03.876311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:23:03.878086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:23:03.899090 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1145/2281] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:23:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:23:13.457785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=6 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:23:13.895948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=56 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:23:56.539779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:23:56.553903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aba4c6-a725-4eee-95a3-f92d39f7ea91 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:23:56.559931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:23:56.560571 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:23:56.560857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:23:56.561134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:23:56.563328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:23:56.638310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:23:56.639904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:23:56.650566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1137/2282] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:23:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:24:02.589785 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:24:02.591516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:24:02.604999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83ea88-41f0-4a13-a320-8825b2b5d511 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:24:02.609545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:24:02.610210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:24:02.610491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:24:02.610773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:24:02.613171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:24:02.628514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:24:02.629980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:24:02.641957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1146/2283] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:24:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:24:06.645560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=33 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:24:12.636702 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:24:56.591363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:24:56.628899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c1c14-a9fb-4d8e-9f69-0f983e6dfeeb service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:24:56.635188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:24:56.635879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:24:56.636167 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:24:56.636440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:24:56.645112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:24:56.646760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:24:56.675898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1138/2284] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:24:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:24:57.191409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:24:57.230366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f31316-33d0-4d7c-8e29-bbe8f308303e service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:24:57.236425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:24:57.237476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:24:57.237764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:24:57.238042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:24:57.247605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:24:57.249257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:24:57.279169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1147/2285] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:24:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:24:57.574988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:24:57.576689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:24:57.611125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e09510-8d50-415f-be8c-fec56f292f44 service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:24:57.615419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:24:57.616101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:24:57.616389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:24:57.616671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:24:57.618823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] Could not find project: 3b5f6238effd43d39cbf0af0f4c582f8.: keystone.exception.ProjectNotFound: Could not find project: 3b5f6238effd43d39cbf0af0f4c582f8. Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jun 17 13:24:57.652043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 13:24:57.653389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self._validate_token_resources() Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 13:24:57.654742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application with Lock( Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._enter() Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.creator() Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application created_value = creator( Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 13:24:57.656263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 13:24:57.657657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 3b5f6238effd43d39cbf0af0f4c582f8. Jun 17 13:24:57.657657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: ERROR keystone.server.flask.application Jun 17 13:24:57.657657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1139/2286] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:24:57 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jun 17 13:24:57.737984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:24:57.748620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f6908-9480-4658-be6a-94eefe59cb8a service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:24:57.752385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:24:57.753171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:24:57.753512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:24:57.753787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:24:57.755713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:24:57.831130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:24:57.832594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:24:57.853316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1148/2287] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:24:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:25:00.466207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:25:00.468999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:00.517631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef0ef-5e00-42f2-bc9a-e2a793f80457 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:25:00.533961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf2ed0-c8a7-4062-8edb-fe7a29a134ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:25:00.535037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf2ed0-c8a7-4062-8edb-fe7a29a134ed service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:00.535190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf2ed0-c8a7-4062-8edb-fe7a29a134ed service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:00.535524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf2ed0-c8a7-4062-8edb-fe7a29a134ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:00.543118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9dbf2ed0-c8a7-4062-8edb-fe7a29a134ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:00.569813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf2ed0-c8a7-4062-8edb-fe7a29a134ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:25:00.572125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf2ed0-c8a7-4062-8edb-fe7a29a134ed service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:25:00.584660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1140/2288] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:25:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:25:00.853931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:25:00.923102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29245ca3-145f-47c3-83e2-a0aa7fa34e49 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:25:00.944374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:25:00.948701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:00.951123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:00.952230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:00.962519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:01.015854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:25:01.017938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:25:01.044807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1149/2289] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:25:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:25:02.596559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77258b-55bd-4ce6-a92f-73d983824fc9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:02.597181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77258b-55bd-4ce6-a92f-73d983824fc9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:02.597181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77258b-55bd-4ce6-a92f-73d983824fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:02.639097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4b77258b-55bd-4ce6-a92f-73d983824fc9 None None] MFA Rules not processed for user `9863e406790b413faf022554ba4b8dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:25:02.641028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4b77258b-55bd-4ce6-a92f-73d983824fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:02.657483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1141/2290] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:25:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:25:03.148923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:25:03.158469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f750e-d2e5-4835-bff7-d038ac4f4ebf service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:25:03.162497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:25:03.163150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:03.163478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:03.163735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:03.165596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:03.175505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:25:03.176938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:25:03.185447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1150/2291] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:25:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:25:03.370755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a6f8-da72-4280-835b-93ea13cb8dc6 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:03.370976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a6f8-da72-4280-835b-93ea13cb8dc6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:03.371256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a6f8-da72-4280-835b-93ea13cb8dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:03.400479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-33f5a6f8-da72-4280-835b-93ea13cb8dc6 None None] MFA Rules not processed for user `a14c1230233e43789087e9239ffb91e2`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:25:03.401728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-33f5a6f8-da72-4280-835b-93ea13cb8dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:03.410413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1142/2292] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:25:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:25:03.608039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:25:03.633425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b56fb2-f7a2-4e1b-a081-6415b3ddf69b service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:25:03.640959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:25:03.641806 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:03.642094 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:03.643775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:03.645952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:03.659893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:25:03.661499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:25:03.672046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1151/2293] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:25:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:25:03.743031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a6f8-da72-4280-835b-93ea13cb8dc6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:25:03.776641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a6f8-da72-4280-835b-93ea13cb8dc6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:25:03.784161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:25:03.785380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:03.785839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:03.787735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:03.793744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:03.815694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:25:03.817178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:25:03.829424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1143/2294] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:25:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:25:04.306113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:25:04.318077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133ad36-8c04-4cec-a505-afa7f9cce21c service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:25:04.325526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514ada0-d27f-4a75-9590-3e8185542f9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:25:04.326434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514ada0-d27f-4a75-9590-3e8185542f9d service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:04.326717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514ada0-d27f-4a75-9590-3e8185542f9d service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:04.328505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514ada0-d27f-4a75-9590-3e8185542f9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:04.340213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514ada0-d27f-4a75-9590-3e8185542f9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:25:04.342897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514ada0-d27f-4a75-9590-3e8185542f9d service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:25:04.360348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1152/2295] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:25:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:25:10.903773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:25:10.922659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76249ad-5648-4129-9ee3-8f1f59774f67 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:25:10.930413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701f0e1-3ff0-44ce-942d-59a566b31ae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:25:10.931282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701f0e1-3ff0-44ce-942d-59a566b31ae4 service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:25:10.931712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701f0e1-3ff0-44ce-942d-59a566b31ae4 service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:25:10.932116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701f0e1-3ff0-44ce-942d-59a566b31ae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:25:10.934403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d701f0e1-3ff0-44ce-942d-59a566b31ae4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:25:10.951331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701f0e1-3ff0-44ce-942d-59a566b31ae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:25:10.953038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701f0e1-3ff0-44ce-942d-59a566b31ae4 service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:25:10.966501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1144/2296] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:25:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:25:14.352689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=88 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:25:20.961861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=69 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:26:54.626816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e167fc-e518-44ad-885b-7f05298aba20 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:26:54.627476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e167fc-e518-44ad-885b-7f05298aba20 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:26:54.627476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e167fc-e518-44ad-885b-7f05298aba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:26:54.658810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a3e167fc-e518-44ad-885b-7f05298aba20 None None] MFA Rules not processed for user `31af58bc06a14e3ba8b709b7deb75341`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:26:54.660083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a3e167fc-e518-44ad-885b-7f05298aba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:26:54.670212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1153/2297] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:26:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:01.569377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b6b6d-3412-4dcd-bfab-2e70719d80de None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:01.569911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b6b6d-3412-4dcd-bfab-2e70719d80de None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:01.570581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b6b6d-3412-4dcd-bfab-2e70719d80de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:01.614480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9a8b6b6d-3412-4dcd-bfab-2e70719d80de None None] MFA Rules not processed for user `f3bfb04b34ed4761bb09be7b29d1ec8f`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:01.616704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9a8b6b6d-3412-4dcd-bfab-2e70719d80de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:01.638305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1145/2298] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:03.368458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:03.369044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:03.369044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:03.387889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] MFA Rules not processed for user `9863e406790b413faf022554ba4b8dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:03.389122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:03.396169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1154/2299] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:08.740308 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4063d6-c1a3-43d3-9043-054dcc54aa07 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:08.740874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4063d6-c1a3-43d3-9043-054dcc54aa07 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:08.740874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4063d6-c1a3-43d3-9043-054dcc54aa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:08.813218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4f4063d6-c1a3-43d3-9043-054dcc54aa07 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:08.814561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4f4063d6-c1a3-43d3-9043-054dcc54aa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:08.880284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1146/2300] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:08.935477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:08.937027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:09.034548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da76913-9a66-4e4b-b7b5-548ab8de8945 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:09.040367 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:09.041046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:09.041418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:09.041663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:09.053530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:09.055140 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:09.067630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1155/2301] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:27:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:27:12.404718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4063d6-c1a3-43d3-9043-054dcc54aa07 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:12.414610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4063d6-c1a3-43d3-9043-054dcc54aa07 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:12.418598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:12.419280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:12.419597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:12.419878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:12.422108 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:12.490979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:12.492515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:12.503781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1147/2302] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:17.325381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:17.343931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93257d-60ff-4174-9b72-759e8c674910 service swift] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:17.353938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:17.358175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:17.359580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:17.360399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:17.363607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:17.394366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:17.397345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:17.416663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1156/2303] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:17.474179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:17.480702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:17.526195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876ff7b-b21a-4771-b352-dabcc0bbfc47 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:17.552161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:17.553697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:17.554205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:17.554711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] PATH_INFO: `/v3/users/efee42b434cd47d99984b85f677983c2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:17.561597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=efee42b434cd47d99984b85f677983c2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:17.569473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:17.679080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:17.691598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efee42b434cd47d99984b85f677983c2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:17.711404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efee42b434cd47d99984b85f677983c2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:17.720377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efee42b434cd47d99984b85f677983c2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:17.734319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efee42b434cd47d99984b85f677983c2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:17.735567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-82733685-f5bd-42dc-ab7f-c05b2af7fe7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efee42b434cd47d99984b85f677983c2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:17.743826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1148/2304] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:17 2022] DELETE /identity/v3/users/efee42b434cd47d99984b85f677983c2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:18.051192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:18.094247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aac58-d05e-4125-b582-034bda141b9a service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:18.244086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:18.246416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:18.246569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:18.246853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] PATH_INFO: `/v3/projects/8ba886227347427bb1cbce939509be05` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:18.248269 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba886227347427bb1cbce939509be05)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:18.250393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:18.339820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:18.348123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba886227347427bb1cbce939509be05', 'request_id': 'req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:18.362767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba886227347427bb1cbce939509be05', 'request_id': 'req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:18.369568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba886227347427bb1cbce939509be05', 'request_id': 'req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:18.394515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] The token cache is being invalidate because project 8ba886227347427bb1cbce939509be05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:18.394829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:18.395553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ebe0f2a0-a358-4746-91c1-115ab49dcc8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:18.405701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1157/2305] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:18 2022] DELETE /identity/v3/projects/8ba886227347427bb1cbce939509be05 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:18.407668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed723-a94b-4e33-ad5a-aea3b6aa5837 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:18.408466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed723-a94b-4e33-ad5a-aea3b6aa5837 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:18.409258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed723-a94b-4e33-ad5a-aea3b6aa5837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:18.497944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b83ed723-a94b-4e33-ad5a-aea3b6aa5837 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:18.499944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b83ed723-a94b-4e33-ad5a-aea3b6aa5837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:18.644214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1149/2306] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:27:18.652993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebe86a-0fc5-4fbb-bae3-be7cb180463a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:18.653339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebe86a-0fc5-4fbb-bae3-be7cb180463a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:18.653587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebe86a-0fc5-4fbb-bae3-be7cb180463a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:18.681564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a9ebe86a-0fc5-4fbb-bae3-be7cb180463a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:18.683411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a9ebe86a-0fc5-4fbb-bae3-be7cb180463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:18.698290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1158/2307] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:18.698845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed723-a94b-4e33-ad5a-aea3b6aa5837 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:18.719248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed723-a94b-4e33-ad5a-aea3b6aa5837 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:18.738845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:18.739641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:18.740136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:18.740571 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:18.742107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:27:18.744541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:18.750452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:18.767497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1150/2308] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 13:27:18.771739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebe86a-0fc5-4fbb-bae3-be7cb180463a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:18.785780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebe86a-0fc5-4fbb-bae3-be7cb180463a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:18.794756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:18.795789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:18.796227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:18.796681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:18.797605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:18.799782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:18.874923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1159/2309] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:18 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:27:18.878484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:18.899663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ad694-af1b-494a-9a65-93c1d63f6e1d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:19.011828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:19.013128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:19.013641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:19.014619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:19.016435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:19.019515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:19.070158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1151/2310] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:18 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:19.074859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:19.089566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b13372-d237-4b87-8f00-6966ccddc3f3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:19.094434 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:19.095129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:19.095450 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:19.095726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:19.102107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:19.105445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:19.125717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1160/2311] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:19.129701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:19.136352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:19.142452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaff54c-c51c-4b53-956a-2a070a53a59d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:19.154360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c778fa-a8b0-41dd-bdbf-fd841db47ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:19.155049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c778fa-a8b0-41dd-bdbf-fd841db47ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:19.155336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c778fa-a8b0-41dd-bdbf-fd841db47ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:19.155674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c778fa-a8b0-41dd-bdbf-fd841db47ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:19.156680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c778fa-a8b0-41dd-bdbf-fd841db47ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:19.158344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c778fa-a8b0-41dd-bdbf-fd841db47ff7 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:19.177682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1152/2312] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:19.184086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:19.201923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43afa5a-fe6b-4799-8d1a-d7e53a753057 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:19.209512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:19.221401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:19.221968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:19.222438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] PATH_INFO: `/v3/projects/253055dcc428497a9b47aee968f37853/users/db86014c66214f07ac865963bcc28bda/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:19.298274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253055dcc428497a9b47aee968f37853, user_id=db86014c66214f07ac865963bcc28bda, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:19.306199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:19.340012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1161/2313] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:19 2022] PUT /identity/v3/projects/253055dcc428497a9b47aee968f37853/users/db86014c66214f07ac865963bcc28bda/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:19.357361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa971de-d1c0-4a23-9f82-56b2ccd497a3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:19.357798 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa971de-d1c0-4a23-9f82-56b2ccd497a3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:19.358215 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa971de-d1c0-4a23-9f82-56b2ccd497a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:19.442460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3aa971de-d1c0-4a23-9f82-56b2ccd497a3 None None] MFA Rules not processed for user `db86014c66214f07ac865963bcc28bda`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:19.444843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3aa971de-d1c0-4a23-9f82-56b2ccd497a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:19.623031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1153/2314] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:19.629985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:19.632020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:19.786642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2a2e3-a4e8-47ac-ad47-4d022db5f8d5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:19.820915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1cdf1-9910-4f72-9ea9-f7cda3b61dac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:19.822076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1cdf1-9910-4f72-9ea9-f7cda3b61dac service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:19.822530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1cdf1-9910-4f72-9ea9-f7cda3b61dac service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:19.823086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1cdf1-9910-4f72-9ea9-f7cda3b61dac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:19.853849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1cdf1-9910-4f72-9ea9-f7cda3b61dac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:19.864842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1cdf1-9910-4f72-9ea9-f7cda3b61dac service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:19.881154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1162/2315] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:20.214936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa971de-d1c0-4a23-9f82-56b2ccd497a3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:20.306330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa971de-d1c0-4a23-9f82-56b2ccd497a3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:20.323084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:20.323954 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:20.324237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:20.324513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] PATH_INFO: `/v3/users/db86014c66214f07ac865963bcc28bda` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:20.328258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db86014c66214f07ac865963bcc28bda)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:20.332941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:20.407704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db86014c66214f07ac865963bcc28bda'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:20.421851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db86014c66214f07ac865963bcc28bda'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:20.430171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db86014c66214f07ac865963bcc28bda'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:20.441421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db86014c66214f07ac865963bcc28bda'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:20.442297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db86014c66214f07ac865963bcc28bda'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:20.446038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:20.453281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1154/2316] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:20 2022] DELETE /identity/v3/users/db86014c66214f07ac865963bcc28bda => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:20.459603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:20.460521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:20.460521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:20.482555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:20.484705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:20.532524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:20.572466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1163/2317] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:27:20.584409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:20.585500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:20.673749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aa318-48de-4327-a248-97e0c9f53755 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:20.678689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13c8af-0091-4389-9116-c76ff189bf3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:20.679644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13c8af-0091-4389-9116-c76ff189bf3d service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:20.680141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13c8af-0091-4389-9116-c76ff189bf3d service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:20.680847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13c8af-0091-4389-9116-c76ff189bf3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:20.702408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13c8af-0091-4389-9116-c76ff189bf3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:20.704726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13c8af-0091-4389-9116-c76ff189bf3d service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:20.725878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1155/2318] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:21.748550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:21.751639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:21.764518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c50d70-af49-478c-9d47-05038d185c44 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:21.770188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:21.771099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:21.771613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:21.771835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] PATH_INFO: `/v3/projects/253055dcc428497a9b47aee968f37853` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:21.773657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] RBAC: Authorizing `identity:delete_project(project_id=253055dcc428497a9b47aee968f37853)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:21.775589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:21.833052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253055dcc428497a9b47aee968f37853', 'request_id': 'req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:21.842163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253055dcc428497a9b47aee968f37853', 'request_id': 'req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:21.851208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253055dcc428497a9b47aee968f37853', 'request_id': 'req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:21.870931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] The token cache is being invalidate because project 253055dcc428497a9b47aee968f37853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:21.871247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:21.871994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0bdfb91e-f2f7-4d9e-906b-d06d71f4285c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:21.873700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1164/2319] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:21 2022] DELETE /identity/v3/projects/253055dcc428497a9b47aee968f37853 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:21.878756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201495a7-24e1-4b85-ad13-15b9b42d1b4d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:21.879276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201495a7-24e1-4b85-ad13-15b9b42d1b4d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:21.879418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201495a7-24e1-4b85-ad13-15b9b42d1b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:21.897238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-201495a7-24e1-4b85-ad13-15b9b42d1b4d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:21.898460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-201495a7-24e1-4b85-ad13-15b9b42d1b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:21.964409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1156/2320] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:21.973969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c35272-18c8-4d70-9ba1-1fe751d5a338 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:21.974337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c35272-18c8-4d70-9ba1-1fe751d5a338 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:21.974679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c35272-18c8-4d70-9ba1-1fe751d5a338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:21.993798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-77c35272-18c8-4d70-9ba1-1fe751d5a338 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:21.995121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-77c35272-18c8-4d70-9ba1-1fe751d5a338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:22.003324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1165/2321] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:22.007847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201495a7-24e1-4b85-ad13-15b9b42d1b4d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.018095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201495a7-24e1-4b85-ad13-15b9b42d1b4d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.022836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.023570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.023841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.024119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.025067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:27:22.025414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.027035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.038835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1157/2322] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:27:22.043867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c35272-18c8-4d70-9ba1-1fe751d5a338 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.053676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c35272-18c8-4d70-9ba1-1fe751d5a338 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.058660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.059413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.059787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.060155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.060931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.062943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.092197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1166/2323] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:22 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:22.095836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.106029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15b938-c212-44fa-b830-4ffe3c09e8a3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.175006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.175637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.175927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.176204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.176920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.178868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.202734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1158/2324] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:22 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:22.203919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.215092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd72ef-35b8-497a-bfad-00ac7d2d5602 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.220661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.221339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.221707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.221989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.222727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.224368 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.236650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1167/2325] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:27:22.240423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.253193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602b17e-f75f-4b72-a713-ee1917084b80 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.259824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136415e-a4b8-410c-b959-c3818d271d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.260405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136415e-a4b8-410c-b959-c3818d271d3a admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.261484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136415e-a4b8-410c-b959-c3818d271d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.261777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136415e-a4b8-410c-b959-c3818d271d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.262708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136415e-a4b8-410c-b959-c3818d271d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.264390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136415e-a4b8-410c-b959-c3818d271d3a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.280728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1159/2326] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:22.282900 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.308561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21febeb-dea7-4864-99ef-da67fd150077 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.323691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.324824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.325251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.325924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] PATH_INFO: `/v3/projects/8e2463ec00b54aa6b24b3b9276d9eb18/users/d87067cd5ee44080a82243c6d75f9b8d/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.362275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2463ec00b54aa6b24b3b9276d9eb18, user_id=d87067cd5ee44080a82243c6d75f9b8d, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.365573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.382103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1168/2327] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:22 2022] PUT /identity/v3/projects/8e2463ec00b54aa6b24b3b9276d9eb18/users/d87067cd5ee44080a82243c6d75f9b8d/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:22.393367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fff39-6629-4745-86f5-a717f850ceeb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.393784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fff39-6629-4745-86f5-a717f850ceeb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.394323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fff39-6629-4745-86f5-a717f850ceeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.417961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:22.430550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-925fff39-6629-4745-86f5-a717f850ceeb None None] MFA Rules not processed for user `d87067cd5ee44080a82243c6d75f9b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:22.432848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-925fff39-6629-4745-86f5-a717f850ceeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:22.511477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1160/2328] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:22.514230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.580959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e689069-8e08-41de-b311-4f8fe18e3b44 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.586691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.587483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.587776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.588100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.588752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.590706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.594294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:22.623528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1169/2329] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:22 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:27:22.624790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fff39-6629-4745-86f5-a717f850ceeb None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.635692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fff39-6629-4745-86f5-a717f850ceeb None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.694552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.695124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.695438 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.695720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.696422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.698445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.720350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1161/2330] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:22 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:22.724751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.738703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362ba59-30b3-443d-bcfe-f6e48a5f8a08 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.745261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.746327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.747689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.748323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.749243 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.752582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.766802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1170/2331] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:22.770669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.788127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba3ea8-21d2-4226-b2be-5571060a3e50 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.801323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b28f9c-1b80-4f63-b8ea-8f69286b1865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.802125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b28f9c-1b80-4f63-b8ea-8f69286b1865 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.802484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b28f9c-1b80-4f63-b8ea-8f69286b1865 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.802828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b28f9c-1b80-4f63-b8ea-8f69286b1865 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.803713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b28f9c-1b80-4f63-b8ea-8f69286b1865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.805396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b28f9c-1b80-4f63-b8ea-8f69286b1865 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.817828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1162/2332] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:27:22.821801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:22.834480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23849c96-c4f7-48d8-a3f6-99ba6d873677 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:22.839783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:22.840805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.841190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.841565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] PATH_INFO: `/v3/projects/e7af32fda8154e7884f2071c44504689/users/3f2981c2fc7843a8a5ba159230943399/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.863590 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7af32fda8154e7884f2071c44504689, user_id=3f2981c2fc7843a8a5ba159230943399, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:22.866353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:22.890564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1171/2333] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:22 2022] PUT /identity/v3/projects/e7af32fda8154e7884f2071c44504689/users/3f2981c2fc7843a8a5ba159230943399/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:22.901767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719412cb-12db-47b9-9ca2-a29b3e587fe1 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:22.902371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719412cb-12db-47b9-9ca2-a29b3e587fe1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:22.902481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719412cb-12db-47b9-9ca2-a29b3e587fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:22.938599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-719412cb-12db-47b9-9ca2-a29b3e587fe1 None None] MFA Rules not processed for user `3f2981c2fc7843a8a5ba159230943399`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:22.940300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-719412cb-12db-47b9-9ca2-a29b3e587fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:23.043753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1163/2334] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:23.048231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:23.144088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc6862-4c3c-492f-963b-bf8b2669e312 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:23.150685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:23.151400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.151720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.152021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.152696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:23.154698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:23.188481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1172/2335] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:23 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:23.192323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719412cb-12db-47b9-9ca2-a29b3e587fe1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:23.215766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719412cb-12db-47b9-9ca2-a29b3e587fe1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:23.221205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:23.297213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:23.298242 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.298531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.298812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.300213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:23.302228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:23.325538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1164/2336] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:23 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:23.329605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:23.340523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df0638-cf55-4a9a-80e1-43c61729160c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:23.345366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:23.346040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.347063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.347063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.347336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:23.349001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:23.361810 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1173/2337] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:23.365921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:23.377191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dcbfa-3863-4669-88d8-90095f035d3d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:23.383870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:23.384874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.385322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.385610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] PATH_INFO: `/v3/projects/e7a4f00821064700aeaa1c3761468a68/users/865c3d4ed3604fb3a65c85222c4c65f1/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.401692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a4f00821064700aeaa1c3761468a68, user_id=865c3d4ed3604fb3a65c85222c4c65f1, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:23.404428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:23.420648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1165/2338] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:23 2022] PUT /identity/v3/projects/e7a4f00821064700aeaa1c3761468a68/users/865c3d4ed3604fb3a65c85222c4c65f1/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:27:23.424561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:23.435063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16bc85-eb81-442b-8dc2-7cbf2ee38a2f admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:23.521728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300ce6f-69da-46fa-9679-c08665798d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:23.522483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ce6f-69da-46fa-9679-c08665798d91 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.522766 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ce6f-69da-46fa-9679-c08665798d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.523683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ce6f-69da-46fa-9679-c08665798d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.524598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300ce6f-69da-46fa-9679-c08665798d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:23.528515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300ce6f-69da-46fa-9679-c08665798d91 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:23.541193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1174/2339] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:23.545427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:23.557104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98780701-25be-49c4-9260-d4af0e162b28 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:23.562787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:23.563867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.564272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.564654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] PATH_INFO: `/v3/projects/e7a4f00821064700aeaa1c3761468a68/users/865c3d4ed3604fb3a65c85222c4c65f1/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.566839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a4f00821064700aeaa1c3761468a68, user_id=865c3d4ed3604fb3a65c85222c4c65f1, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:23.570077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:23.581605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1166/2340] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:23 2022] PUT /identity/v3/projects/e7a4f00821064700aeaa1c3761468a68/users/865c3d4ed3604fb3a65c85222c4c65f1/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:23.593541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.593991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.594390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.622677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] MFA Rules not processed for user `865c3d4ed3604fb3a65c85222c4c65f1`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:23.624389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:23.632998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:23.704506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1175/2341] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:23.715990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:23.717821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:23.774630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0beeb-9954-4d99-8b64-96402650032d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:23.778929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:23.779605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:23.779889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:23.780169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:23.789376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:23.790929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:23.800688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1167/2342] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:24.338663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:24.340357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:24.352879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab9d1-063f-4a47-bf23-e636f1333678 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:24.357529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3527ee6-2ede-4093-b9e7-107c5206d092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:24.358218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3527ee6-2ede-4093-b9e7-107c5206d092 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:24.358852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3527ee6-2ede-4093-b9e7-107c5206d092 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:24.359123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3527ee6-2ede-4093-b9e7-107c5206d092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:24.418095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3527ee6-2ede-4093-b9e7-107c5206d092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:24.419592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3527ee6-2ede-4093-b9e7-107c5206d092 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:24.428826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1176/2343] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:28.476621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:28.485849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47671d7f-c0d8-4205-8f54-269b280c4f41 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:28.489837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431fca8-cc93-4696-ac63-e42541eecba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:28.490518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431fca8-cc93-4696-ac63-e42541eecba6 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:28.490792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431fca8-cc93-4696-ac63-e42541eecba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:28.491069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431fca8-cc93-4696-ac63-e42541eecba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:28.558591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431fca8-cc93-4696-ac63-e42541eecba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:28.560204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431fca8-cc93-4696-ac63-e42541eecba6 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:28.571305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:28.578891 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1168/2344] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:28.747303 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce975e-be5e-4a19-986e-d6df91d8f448 None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:28.747931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce975e-be5e-4a19-986e-d6df91d8f448 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:28.748157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce975e-be5e-4a19-986e-d6df91d8f448 None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:28.749599 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1177/2345] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 13:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 13:27:28.756257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8646d-e412-45ba-9cfc-614bef16c58d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:28.756510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8646d-e412-45ba-9cfc-614bef16c58d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:28.756731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8646d-e412-45ba-9cfc-614bef16c58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:28.803785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-bbe8646d-e412-45ba-9cfc-614bef16c58d None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:28.805012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bbe8646d-e412-45ba-9cfc-614bef16c58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:28.913917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1169/2346] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:27:28.926851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce975e-be5e-4a19-986e-d6df91d8f448 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:28.928956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5cce975e-be5e-4a19-986e-d6df91d8f448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:29.018911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce975e-be5e-4a19-986e-d6df91d8f448 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:29.023432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbdaba-e220-4cad-b571-f1fb8703304e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:29.024105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbdaba-e220-4cad-b571-f1fb8703304e service glance] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:29.024393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbdaba-e220-4cad-b571-f1fb8703304e service glance] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:29.024667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbdaba-e220-4cad-b571-f1fb8703304e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:29.036720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbdaba-e220-4cad-b571-f1fb8703304e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:29.038523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbdaba-e220-4cad-b571-f1fb8703304e service glance] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:29.050895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1178/2347] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:29.130433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798da65-93f8-420a-98cc-c3d0eca91420 None None] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:29.130777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798da65-93f8-420a-98cc-c3d0eca91420 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:29.131056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798da65-93f8-420a-98cc-c3d0eca91420 None None] PATH_INFO: `/` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:29.132746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1170/2348] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 13:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 13:27:29.143769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98f163-b1ae-416f-bbb1-ad750c9557ce None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:29.144113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98f163-b1ae-416f-bbb1-ad750c9557ce None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:29.144539 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98f163-b1ae-416f-bbb1-ad750c9557ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:29.185743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4a98f163-b1ae-416f-bbb1-ad750c9557ce None None] MFA Rules not processed for user `40f6ae0e5e164461b4c51b829be9a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:29.186981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4a98f163-b1ae-416f-bbb1-ad750c9557ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:29.195828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:29.246280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1179/2349] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 13:27:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:29.258746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798da65-93f8-420a-98cc-c3d0eca91420 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:29.260682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9798da65-93f8-420a-98cc-c3d0eca91420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:29.283587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798da65-93f8-420a-98cc-c3d0eca91420 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:29.287693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d7da1-69a0-440c-bf64-4f75afcac4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:29.288442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d7da1-69a0-440c-bf64-4f75afcac4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:29.288792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d7da1-69a0-440c-bf64-4f75afcac4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:29.289134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d7da1-69a0-440c-bf64-4f75afcac4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:29.299668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d7da1-69a0-440c-bf64-4f75afcac4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:29.301124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d7da1-69a0-440c-bf64-4f75afcac4e5 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:29.312012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1171/2350] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:29.408032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1260fd9-9d7f-4bdf-9288-4325588d7dee None None] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:29.408306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1260fd9-9d7f-4bdf-9288-4325588d7dee None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:29.408620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1260fd9-9d7f-4bdf-9288-4325588d7dee None None] PATH_INFO: `/` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:29.410217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1180/2351] 10.222.0.87 () {54 vars in 921 bytes} [Fri Jun 17 13:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 13:27:29.416780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe9167-1a1c-4cda-9c9e-56bccd82aca3 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:29.417306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe9167-1a1c-4cda-9c9e-56bccd82aca3 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:29.417753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe9167-1a1c-4cda-9c9e-56bccd82aca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:29.441149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-1fbe9167-1a1c-4cda-9c9e-56bccd82aca3 None None] MFA Rules not processed for user `abaa59b4e88642c681e9563295f6fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:29.442406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1fbe9167-1a1c-4cda-9c9e-56bccd82aca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:29.455190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1172/2352] 10.222.0.87 () {58 vars in 1018 bytes} [Fri Jun 17 13:27:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:29.467181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1260fd9-9d7f-4bdf-9288-4325588d7dee None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:29.469130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c1260fd9-9d7f-4bdf-9288-4325588d7dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:29.484567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1260fd9-9d7f-4bdf-9288-4325588d7dee None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:29.488951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810617d-7842-4a3d-8a3d-c8c6f0658e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:29.489623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810617d-7842-4a3d-8a3d-c8c6f0658e00 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:29.489905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810617d-7842-4a3d-8a3d-c8c6f0658e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:29.490176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810617d-7842-4a3d-8a3d-c8c6f0658e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:29.502468 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810617d-7842-4a3d-8a3d-c8c6f0658e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:29.503973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810617d-7842-4a3d-8a3d-c8c6f0658e00 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:29.516302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1181/2353] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:30.843095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe9167-1a1c-4cda-9c9e-56bccd82aca3 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:30.911321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe9167-1a1c-4cda-9c9e-56bccd82aca3 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:30.916126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:30.917180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:30.917462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:30.917711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] PATH_INFO: `/v3/users/d87067cd5ee44080a82243c6d75f9b8d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:30.919029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87067cd5ee44080a82243c6d75f9b8d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:30.921028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:30.991847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87067cd5ee44080a82243c6d75f9b8d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.007636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:31.011662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87067cd5ee44080a82243c6d75f9b8d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.019358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87067cd5ee44080a82243c6d75f9b8d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.028846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87067cd5ee44080a82243c6d75f9b8d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.029452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87067cd5ee44080a82243c6d75f9b8d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.036460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1173/2354] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:30 2022] DELETE /identity/v3/users/d87067cd5ee44080a82243c6d75f9b8d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:31.041032 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:31.041768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:31.042049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:31.061929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:31.063336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:31.145081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1182/2355] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:31.157303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:31.159041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:31.232282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9b5b5-83f2-4253-a721-93cb73a5753f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:31.236529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:31.237205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:31.237508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:31.237745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:31.248400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:31.249902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:31.260884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1174/2356] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:31.595393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:31.596875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:31.610744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742da17-1da7-47d3-8b95-5572bf0390fa None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:31.616777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:31.617672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:31.617935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:31.618230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] PATH_INFO: `/v3/projects/8e2463ec00b54aa6b24b3b9276d9eb18` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:31.620208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2463ec00b54aa6b24b3b9276d9eb18)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:31.622196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:31.686764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2463ec00b54aa6b24b3b9276d9eb18', 'request_id': 'req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.693190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2463ec00b54aa6b24b3b9276d9eb18', 'request_id': 'req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.699408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2463ec00b54aa6b24b3b9276d9eb18', 'request_id': 'req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.714144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] The token cache is being invalidate because project 8e2463ec00b54aa6b24b3b9276d9eb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:31.714438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.715095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.716725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1183/2357] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:31 2022] DELETE /identity/v3/projects/8e2463ec00b54aa6b24b3b9276d9eb18 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:27:31.720849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:31.732045 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dd6bc-0371-45fe-b163-714a7facc359 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:31.796554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:31.797465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:31.797745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:31.800030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] PATH_INFO: `/v3/users/3f2981c2fc7843a8a5ba159230943399` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:31.801348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2981c2fc7843a8a5ba159230943399)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:31.804289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:31.868848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2981c2fc7843a8a5ba159230943399'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.882150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2981c2fc7843a8a5ba159230943399'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.889600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2981c2fc7843a8a5ba159230943399'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.897549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2981c2fc7843a8a5ba159230943399'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.898203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2981c2fc7843a8a5ba159230943399'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:31.900542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:31.905795 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1175/2358] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:31 2022] DELETE /identity/v3/users/3f2981c2fc7843a8a5ba159230943399 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:32.220514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:32.225853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:32.243692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64e0bd-8dc3-4a2b-bcb4-c8bd31b5b5f9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:32.282564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:32.319253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:32.320196 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:32.320479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:32.320756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] PATH_INFO: `/v3/projects/e7af32fda8154e7884f2071c44504689` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:32.322082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7af32fda8154e7884f2071c44504689)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:32.324393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:32.384891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7af32fda8154e7884f2071c44504689', 'request_id': 'req-f0e33c15-faba-4987-ba5d-c784197b9e02'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.392246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7af32fda8154e7884f2071c44504689', 'request_id': 'req-f0e33c15-faba-4987-ba5d-c784197b9e02'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.399068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7af32fda8154e7884f2071c44504689', 'request_id': 'req-f0e33c15-faba-4987-ba5d-c784197b9e02'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.415515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] The token cache is being invalidate because project e7af32fda8154e7884f2071c44504689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:32.415795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.416550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.422431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1184/2359] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:32 2022] DELETE /identity/v3/projects/e7af32fda8154e7884f2071c44504689 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:32.424302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:32.427228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:32.441296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ea5cc-732f-4f95-b5f9-a9451395612f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:32.540844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:32.541874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:32.542273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:32.542548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] PATH_INFO: `/v3/users/865c3d4ed3604fb3a65c85222c4c65f1` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:32.544203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=865c3d4ed3604fb3a65c85222c4c65f1)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:32.556507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:32.624687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865c3d4ed3604fb3a65c85222c4c65f1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.640044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865c3d4ed3604fb3a65c85222c4c65f1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.647285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865c3d4ed3604fb3a65c85222c4c65f1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.655698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865c3d4ed3604fb3a65c85222c4c65f1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.656379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-548f6589-05f1-452e-b05e-52f3c61af223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865c3d4ed3604fb3a65c85222c4c65f1'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:32.663336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1176/2360] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:32 2022] DELETE /identity/v3/users/865c3d4ed3604fb3a65c85222c4c65f1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:32.964956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:32.966997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:32.986495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33c15-faba-4987-ba5d-c784197b9e02 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.049837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.050391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.050391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.069473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:33.070373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.070863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:33.071273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.071595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.071877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] PATH_INFO: `/v3/projects/e7a4f00821064700aeaa1c3761468a68` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.073844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a4f00821064700aeaa1c3761468a68)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.076178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.138078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1177/2361] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 13:27:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:33.141832 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a4f00821064700aeaa1c3761468a68', 'request_id': 'req-2c30a5de-9822-4950-bdd3-e87fe952d45d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:33.150036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a4f00821064700aeaa1c3761468a68', 'request_id': 'req-2c30a5de-9822-4950-bdd3-e87fe952d45d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:33.153836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:33.155296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:33.157709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a4f00821064700aeaa1c3761468a68', 'request_id': 'req-2c30a5de-9822-4950-bdd3-e87fe952d45d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:33.161077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:33.180208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] The token cache is being invalidate because project e7a4f00821064700aeaa1c3761468a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:33.180812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:33.181531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2c30a5de-9822-4950-bdd3-e87fe952d45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:33.183703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1185/2362] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:32 2022] DELETE /identity/v3/projects/e7a4f00821064700aeaa1c3761468a68 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:33.190933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5f2b3-64cf-4842-b8b6-b314fd666dad None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.191306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5f2b3-64cf-4842-b8b6-b314fd666dad None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.191687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5f2b3-64cf-4842-b8b6-b314fd666dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.196137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:33.215988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-72e5f2b3-64cf-4842-b8b6-b314fd666dad None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:33.217443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-72e5f2b3-64cf-4842-b8b6-b314fd666dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:33.246207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6fefe-07e2-43d6-9fd4-ead66873ce56 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.304481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1186/2363] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:27:33.313229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4afae-78fe-4cb6-a88d-254e83b0f40f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.313493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4afae-78fe-4cb6-a88d-254e83b0f40f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.313773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4afae-78fe-4cb6-a88d-254e83b0f40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.328116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.329353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.329689 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.329970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.332204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:33.335855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-bdf4afae-78fe-4cb6-a88d-254e83b0f40f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:33.337116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-bdf4afae-78fe-4cb6-a88d-254e83b0f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:33.354882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1187/2364] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:33.355927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4afae-78fe-4cb6-a88d-254e83b0f40f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:33.371249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4afae-78fe-4cb6-a88d-254e83b0f40f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.376018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.376702 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.376979 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.377249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.377989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:27:33.378292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.379953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.395215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1188/2365] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:27:33.401147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:33.411614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2ada5-adc9-4c7a-8d8e-bb8494e5e16c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.412880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.414577 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.417382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.418068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.418356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.418634 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.419411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.421988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.429443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1178/2366] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:27:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 13:27:33.467250 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1189/2367] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:33 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:27:33.467451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:33.482389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b872-8ead-42ec-aa7a-228cd2927ddc service swift] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.587186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.588746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.589878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.591146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.592819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.601844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.624618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:33.650157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1179/2368] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:33 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:33.655190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:33.670340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdc556-7189-42d9-9214-7956f8d8806e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.683576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.684331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.684616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.684893 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.685642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.687293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.714052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1190/2369] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:33.715301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:33.728236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4adb1-bfa1-46ee-804b-5dbf928ae87b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.747384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a4ca-8637-4523-aafe-98720d810eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.748272 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a4ca-8637-4523-aafe-98720d810eea admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.748566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a4ca-8637-4523-aafe-98720d810eea admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.748906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a4ca-8637-4523-aafe-98720d810eea admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.749720 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a4ca-8637-4523-aafe-98720d810eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.751552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a4ca-8637-4523-aafe-98720d810eea admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.774441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1180/2370] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:33.788236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:33.813672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4728c3-cd93-4c15-8b2f-598384b5e21d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:33.824106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:33.824106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.824604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.824604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] PATH_INFO: `/v3/projects/1e7cb1e068554cf7b9a3cae025457b6a/users/0a8ab30c87fd4a33af248baf3966c61e/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.857763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7cb1e068554cf7b9a3cae025457b6a, user_id=0a8ab30c87fd4a33af248baf3966c61e, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:33.863313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:33.896310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1191/2371] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:33 2022] PUT /identity/v3/projects/1e7cb1e068554cf7b9a3cae025457b6a/users/0a8ab30c87fd4a33af248baf3966c61e/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:33.918160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d7984-707b-4592-96b1-5c26432e06f5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:33.918656 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d7984-707b-4592-96b1-5c26432e06f5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:33.919036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d7984-707b-4592-96b1-5c26432e06f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:33.971299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-735d7984-707b-4592-96b1-5c26432e06f5 None None] MFA Rules not processed for user `0a8ab30c87fd4a33af248baf3966c61e`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:33.972740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-735d7984-707b-4592-96b1-5c26432e06f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:34.068726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1181/2372] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:34.074008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.152653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5541c-c0ea-4d58-b588-c5d6e7f89928 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.158078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.159099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.159493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.159846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.160784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:34.163061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:34.195406 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1192/2373] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:34 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:27:34.198195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d7984-707b-4592-96b1-5c26432e06f5 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.209325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d7984-707b-4592-96b1-5c26432e06f5 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.291543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.291543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.291543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.291543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.292122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:34.292122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:34.316989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1182/2374] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:34 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:34.321127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.327024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:34.333328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3feaeaa-a788-4d63-a3e3-38c314dcb098 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.341877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.342568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.342850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.343126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.343912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:34.345657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:34.362193 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1193/2375] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:34.364500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.375899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb8be-b180-4d31-b9de-4858403a89fc admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.381467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.382274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.382552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.382827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] PATH_INFO: `/v3/projects/37e3399b80db4808922122adca69e169/users/9f94a1eb8fe74ab9ac616bd498489189/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.398548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e3399b80db4808922122adca69e169, user_id=9f94a1eb8fe74ab9ac616bd498489189, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:34.401562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:34.415910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1183/2376] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:34 2022] PUT /identity/v3/projects/37e3399b80db4808922122adca69e169/users/9f94a1eb8fe74ab9ac616bd498489189/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:34.420165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.430965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10880953-17fe-4fa8-bfa3-58d9e182a63c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.497566 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e9e8b-fffa-42e3-939a-6850ccb66424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.498493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9e8b-fffa-42e3-939a-6850ccb66424 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.499051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9e8b-fffa-42e3-939a-6850ccb66424 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.499555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9e8b-fffa-42e3-939a-6850ccb66424 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.500391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e9e8b-fffa-42e3-939a-6850ccb66424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:34.502036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e9e8b-fffa-42e3-939a-6850ccb66424 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:34.513623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1194/2377] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:34.518120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.528726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d063c9-5eac-48b2-900b-e64f0cb7bffc admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.533257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.534139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.534419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.534692 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] PATH_INFO: `/v3/projects/37e3399b80db4808922122adca69e169/users/9f94a1eb8fe74ab9ac616bd498489189/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.536654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e3399b80db4808922122adca69e169, user_id=9f94a1eb8fe74ab9ac616bd498489189, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:34.539348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:34.550271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1184/2378] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:34 2022] PUT /identity/v3/projects/37e3399b80db4808922122adca69e169/users/9f94a1eb8fe74ab9ac616bd498489189/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:34.561289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d43f-31a5-407e-bddf-a148c18f6e6f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.561578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d43f-31a5-407e-bddf-a148c18f6e6f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.561854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d43f-31a5-407e-bddf-a148c18f6e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.592175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3185d43f-31a5-407e-bddf-a148c18f6e6f None None] MFA Rules not processed for user `9f94a1eb8fe74ab9ac616bd498489189`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:34.593397 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3185d43f-31a5-407e-bddf-a148c18f6e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:34.670324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1195/2379] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:34.683617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.685899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:34.722966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:34.753560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585af30-15c3-4034-8528-863e0735f23e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.757455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.758131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.758419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.758708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:34.768728 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:34.770211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:34.780502 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1185/2380] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:34.903589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185d43f-31a5-407e-bddf-a148c18f6e6f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:34.916334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185d43f-31a5-407e-bddf-a148c18f6e6f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:34.920764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3850b80-d0b6-424a-b615-0637f04d8b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:34.921351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3850b80-d0b6-424a-b615-0637f04d8b33 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:34.921635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3850b80-d0b6-424a-b615-0637f04d8b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:34.921914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3850b80-d0b6-424a-b615-0637f04d8b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:35.026304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3850b80-d0b6-424a-b615-0637f04d8b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:35.027881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3850b80-d0b6-424a-b615-0637f04d8b33 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:35.044659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1196/2381] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:37.715846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:37.782285 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f0a4f-0d25-4a08-873e-31656cb09d4a service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:37.786883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:37.787817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:37.788102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:37.788381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] PATH_INFO: `/v3/users/0a8ab30c87fd4a33af248baf3966c61e` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:37.789606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a8ab30c87fd4a33af248baf3966c61e)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:37.791939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:37.865276 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a8ab30c87fd4a33af248baf3966c61e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:37.879684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a8ab30c87fd4a33af248baf3966c61e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:37.888022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a8ab30c87fd4a33af248baf3966c61e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:37.897422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a8ab30c87fd4a33af248baf3966c61e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:37.898432 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a8ab30c87fd4a33af248baf3966c61e'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:37.908979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1186/2382] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:37 2022] DELETE /identity/v3/users/0a8ab30c87fd4a33af248baf3966c61e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:37.910850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:37.911419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:37.911419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:37.929684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:37.930911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:37.937471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:38.012065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1197/2383] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:38.022133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:38.023715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:38.092056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900dffd-f7b5-4a52-a0de-d32d4e27f7cc admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:38.095890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:38.096560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:38.096842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:38.097117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:38.103954 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:38.113587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:38.115009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:38.126627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1187/2384] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:38.514466 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:38.516384 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:38.532307 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c669-0d3f-4752-aaac-074acd5e5405 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:38.549311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:38.550954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:38.551596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:38.552136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] PATH_INFO: `/v3/projects/1e7cb1e068554cf7b9a3cae025457b6a` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:38.554414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7cb1e068554cf7b9a3cae025457b6a)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:38.560765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:38.636467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7cb1e068554cf7b9a3cae025457b6a', 'request_id': 'req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.665314 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7cb1e068554cf7b9a3cae025457b6a', 'request_id': 'req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.673567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7cb1e068554cf7b9a3cae025457b6a', 'request_id': 'req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.719101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] The token cache is being invalidate because project 1e7cb1e068554cf7b9a3cae025457b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:38.720267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.721121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.723155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1198/2385] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:38 2022] DELETE /identity/v3/projects/1e7cb1e068554cf7b9a3cae025457b6a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:38.727418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:38.747248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fab76f-b1c3-4031-9b67-19d5c3a81c52 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:38.822455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:38.823558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:38.823844 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:38.824126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] PATH_INFO: `/v3/users/9f94a1eb8fe74ab9ac616bd498489189` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:38.826459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f94a1eb8fe74ab9ac616bd498489189)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:38.828472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:38.900090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f94a1eb8fe74ab9ac616bd498489189'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.919092 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f94a1eb8fe74ab9ac616bd498489189'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.927010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f94a1eb8fe74ab9ac616bd498489189'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.937436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f94a1eb8fe74ab9ac616bd498489189'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.938164 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a6bf54ff-0ca4-466a-a98c-5a17b4b5426e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f94a1eb8fe74ab9ac616bd498489189'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:38.945199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1188/2386] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:38 2022] DELETE /identity/v3/users/9f94a1eb8fe74ab9ac616bd498489189 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:39.261486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:39.263152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:39.289981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86778f17-eeaf-48f5-8bd6-ac43b4d787cf admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:39.369003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:39.370666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.371028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.371438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] PATH_INFO: `/v3/projects/37e3399b80db4808922122adca69e169` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.372914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e3399b80db4808922122adca69e169)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:39.375156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:39.428755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:39.441548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e3399b80db4808922122adca69e169', 'request_id': 'req-debad66a-ece8-479d-a350-2e3c91f70648'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:39.452418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e3399b80db4808922122adca69e169', 'request_id': 'req-debad66a-ece8-479d-a350-2e3c91f70648'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:39.465393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e3399b80db4808922122adca69e169', 'request_id': 'req-debad66a-ece8-479d-a350-2e3c91f70648'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:39.475113 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] The token cache is being invalidate because project 37e3399b80db4808922122adca69e169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:39.475482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:39.476302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-debad66a-ece8-479d-a350-2e3c91f70648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:39.477831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1199/2387] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:39 2022] DELETE /identity/v3/projects/37e3399b80db4808922122adca69e169 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:39.484275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8cab6-9e0f-43f1-ad82-886ad1e71ffb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.484564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8cab6-9e0f-43f1-ad82-886ad1e71ffb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.484837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8cab6-9e0f-43f1-ad82-886ad1e71ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.504664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-10b8cab6-9e0f-43f1-ad82-886ad1e71ffb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:39.505883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-10b8cab6-9e0f-43f1-ad82-886ad1e71ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:39.580152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1189/2388] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:39.589980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e39f-e76d-4640-ad5d-413e2ed45c99 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.590380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e39f-e76d-4640-ad5d-413e2ed45c99 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.590853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e39f-e76d-4640-ad5d-413e2ed45c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.612540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4715e39f-e76d-4640-ad5d-413e2ed45c99 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:39.613963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4715e39f-e76d-4640-ad5d-413e2ed45c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:39.623287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1200/2389] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:39.627719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8cab6-9e0f-43f1-ad82-886ad1e71ffb None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:39.637861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8cab6-9e0f-43f1-ad82-886ad1e71ffb None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:39.642594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:39.643284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.643603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.643883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.644664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:27:39.644976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:39.646624 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:39.657300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:39.661410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1190/2390] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:27:39.667299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e39f-e76d-4640-ad5d-413e2ed45c99 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:39.682231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e39f-e76d-4640-ad5d-413e2ed45c99 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:39.687464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:39.688180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.688486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.688765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.689480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:39.691443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:39.718891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1201/2391] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:39 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:39.723287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:39.734550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a73e9-d189-4b08-9d2e-5167f6b19c49 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:39.797993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:39.800022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.800493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.800942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.801823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:39.805160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:39.849726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1191/2392] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:39 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:39.851285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:39.864206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb4aee-4198-436c-aae2-72414861a8c5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:39.871233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:39.871968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.872253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.872535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.873293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:39.874948 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:39.896309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1202/2393] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:39.899295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:39.922930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ab322-90df-456b-a80c-19720f3f7419 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:39.929365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5337ff-82cb-4dbc-8c6d-8f45545c732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:39.930464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5337ff-82cb-4dbc-8c6d-8f45545c732c admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.931001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5337ff-82cb-4dbc-8c6d-8f45545c732c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.931566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5337ff-82cb-4dbc-8c6d-8f45545c732c admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.932779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5337ff-82cb-4dbc-8c6d-8f45545c732c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:39.935134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5337ff-82cb-4dbc-8c6d-8f45545c732c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:39.948985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1192/2394] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:39.953331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:39.964007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6da6a7-06f6-4bd6-ad84-71b72eb82ba6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:39.968978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:39.969902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:39.970181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:39.970454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] PATH_INFO: `/v3/projects/2e8db16ae9024d3da161f0e5d0fc9da5/users/81096f806cbf43ffb4e60f8d149c64dc/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:39.987593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8db16ae9024d3da161f0e5d0fc9da5, user_id=81096f806cbf43ffb4e60f8d149c64dc, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:39.990863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.007759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1203/2395] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:39 2022] PUT /identity/v3/projects/2e8db16ae9024d3da161f0e5d0fc9da5/users/81096f806cbf43ffb4e60f8d149c64dc/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:40.019789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500a0b5-3d9a-4bd1-8fce-46d108ecfdfd None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.020152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500a0b5-3d9a-4bd1-8fce-46d108ecfdfd None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.020499 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500a0b5-3d9a-4bd1-8fce-46d108ecfdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.051329 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9500a0b5-3d9a-4bd1-8fce-46d108ecfdfd None None] MFA Rules not processed for user `81096f806cbf43ffb4e60f8d149c64dc`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:40.052581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9500a0b5-3d9a-4bd1-8fce-46d108ecfdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:40.156141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1193/2396] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:40.158845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:40.235665 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eaf6e-5ce0-4fcf-96b0-763cbf45a6a5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:40.240477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:40.241201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.241570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.241875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.242668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:40.245066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.274782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1204/2397] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:40 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:40.279532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500a0b5-3d9a-4bd1-8fce-46d108ecfdfd None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:40.293831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500a0b5-3d9a-4bd1-8fce-46d108ecfdfd None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:40.367113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:40.367829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.368140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.368412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.369469 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:40.371748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.388760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:40.395867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1194/2398] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:40 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:40.399782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:40.413845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986835-d456-447b-bfd0-d4b56d83153c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:40.418950 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:40.419754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.420103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.420441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.421263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:40.422946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.434260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1205/2399] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:40.438552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:40.449414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687114-7738-4c9c-ab7f-2019fa3a3546 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:40.457346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:40.458268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.458556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.458831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] PATH_INFO: `/v3/projects/80fd83cbc2fb4a6ab9430030122728bd/users/371fe7beb503424494c9564d02cd2fe6/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.476964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80fd83cbc2fb4a6ab9430030122728bd, user_id=371fe7beb503424494c9564d02cd2fe6, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:40.488425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.502927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1195/2400] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:40 2022] PUT /identity/v3/projects/80fd83cbc2fb4a6ab9430030122728bd/users/371fe7beb503424494c9564d02cd2fe6/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:40.506861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:40.518030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80825f34-185a-4159-8d71-0ea929d9b0d7 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:40.582824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:40.618906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2bac-c11a-4a6e-9201-890c17e671d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:40.620114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2bac-c11a-4a6e-9201-890c17e671d4 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.620524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2bac-c11a-4a6e-9201-890c17e671d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.620890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2bac-c11a-4a6e-9201-890c17e671d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.621712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2bac-c11a-4a6e-9201-890c17e671d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:40.623513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2bac-c11a-4a6e-9201-890c17e671d4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.634515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1206/2401] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:40.638925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:40.650204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97c395-ee05-42bc-bca9-72569bd9bdd3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:40.656096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:40.657104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.657446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.657787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] PATH_INFO: `/v3/projects/80fd83cbc2fb4a6ab9430030122728bd/users/371fe7beb503424494c9564d02cd2fe6/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.659931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80fd83cbc2fb4a6ab9430030122728bd, user_id=371fe7beb503424494c9564d02cd2fe6, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:40.662753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.674758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1196/2402] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:40 2022] PUT /identity/v3/projects/80fd83cbc2fb4a6ab9430030122728bd/users/371fe7beb503424494c9564d02cd2fe6/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:40.685548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e09a4a-25e1-4ee7-96d1-57b2ffbbe0bd None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.685836 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e09a4a-25e1-4ee7-96d1-57b2ffbbe0bd None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.686112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e09a4a-25e1-4ee7-96d1-57b2ffbbe0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.721654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-68e09a4a-25e1-4ee7-96d1-57b2ffbbe0bd None None] MFA Rules not processed for user `371fe7beb503424494c9564d02cd2fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:40.722952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-68e09a4a-25e1-4ee7-96d1-57b2ffbbe0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:40.803362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1207/2403] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:40.812252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:40.814002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:40.888323 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d44680-c1c2-4b1c-92d5-8ceffab24d9b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:40.892105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8863a-6b32-4de1-a1be-e7b25dbcc1e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:40.892785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8863a-6b32-4de1-a1be-e7b25dbcc1e9 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:40.893067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8863a-6b32-4de1-a1be-e7b25dbcc1e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:40.893342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8863a-6b32-4de1-a1be-e7b25dbcc1e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:40.957003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8863a-6b32-4de1-a1be-e7b25dbcc1e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:40.958837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8863a-6b32-4de1-a1be-e7b25dbcc1e9 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:40.969823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1197/2404] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:43.630675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e09a4a-25e1-4ee7-96d1-57b2ffbbe0bd None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:43.700562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e09a4a-25e1-4ee7-96d1-57b2ffbbe0bd None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:43.705534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:43.706410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:43.706694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:43.706969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] PATH_INFO: `/v3/users/81096f806cbf43ffb4e60f8d149c64dc` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:43.708195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81096f806cbf43ffb4e60f8d149c64dc)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:43.710122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:43.762496 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81096f806cbf43ffb4e60f8d149c64dc'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:43.770807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:43.776063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81096f806cbf43ffb4e60f8d149c64dc'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:43.785106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81096f806cbf43ffb4e60f8d149c64dc'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:43.793374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81096f806cbf43ffb4e60f8d149c64dc'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:43.794141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81096f806cbf43ffb4e60f8d149c64dc'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:43.800449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1208/2405] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:43 2022] DELETE /identity/v3/users/81096f806cbf43ffb4e60f8d149c64dc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:43.805536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:43.806084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:43.806084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:43.825218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:43.826448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:43.892890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1198/2406] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:27:43.902841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:43.904845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:43.970913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520515-138d-4b8f-b8b9-ed454c170570 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:43.975293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:43.975995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:43.976275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:43.976551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:43.987690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:43.989156 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:43.999415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1209/2407] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:44.282779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:44.284426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:44.299021 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e908e25-043e-438f-950c-0986459a7d9a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:44.304466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:44.305370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:44.305660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:44.305961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] PATH_INFO: `/v3/projects/2e8db16ae9024d3da161f0e5d0fc9da5` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:44.307414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8db16ae9024d3da161f0e5d0fc9da5)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:44.309424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:44.321298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:44.361076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8db16ae9024d3da161f0e5d0fc9da5', 'request_id': 'req-21d97732-2e71-43bf-b245-41df0a4aa73a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.367690 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8db16ae9024d3da161f0e5d0fc9da5', 'request_id': 'req-21d97732-2e71-43bf-b245-41df0a4aa73a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.374592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8db16ae9024d3da161f0e5d0fc9da5', 'request_id': 'req-21d97732-2e71-43bf-b245-41df0a4aa73a'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.387884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] The token cache is being invalidate because project 2e8db16ae9024d3da161f0e5d0fc9da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:44.388156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.388801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.390380 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1199/2408] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:44 2022] DELETE /identity/v3/projects/2e8db16ae9024d3da161f0e5d0fc9da5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:44.394501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:44.404978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb5c27-d530-4a94-9c88-533298e259ba service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:44.466764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:44.467701 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:44.467990 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:44.468271 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] PATH_INFO: `/v3/users/371fe7beb503424494c9564d02cd2fe6` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:44.469534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371fe7beb503424494c9564d02cd2fe6)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:44.471523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:44.520748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371fe7beb503424494c9564d02cd2fe6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.533703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371fe7beb503424494c9564d02cd2fe6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.543350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371fe7beb503424494c9564d02cd2fe6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.551221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371fe7beb503424494c9564d02cd2fe6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.551987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-bfa8a484-da27-490e-bc26-c71aa0863272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371fe7beb503424494c9564d02cd2fe6'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:44.553501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:44.559274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1210/2409] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:44 2022] DELETE /identity/v3/users/371fe7beb503424494c9564d02cd2fe6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:44.933162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:44.934664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:44.950870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97732-2e71-43bf-b245-41df0a4aa73a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.011753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.012663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.012945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.013205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] PATH_INFO: `/v3/projects/80fd83cbc2fb4a6ab9430030122728bd` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.014439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80fd83cbc2fb4a6ab9430030122728bd)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.016474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.070104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80fd83cbc2fb4a6ab9430030122728bd', 'request_id': 'req-1cf6e0ad-1940-4c30-ba64-ffd57734304b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:45.083901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80fd83cbc2fb4a6ab9430030122728bd', 'request_id': 'req-1cf6e0ad-1940-4c30-ba64-ffd57734304b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:45.089714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80fd83cbc2fb4a6ab9430030122728bd', 'request_id': 'req-1cf6e0ad-1940-4c30-ba64-ffd57734304b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:45.103838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] The token cache is being invalidate because project 80fd83cbc2fb4a6ab9430030122728bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:45.104128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:45.104760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1cf6e0ad-1940-4c30-ba64-ffd57734304b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:45.106337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1200/2410] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:44 2022] DELETE /identity/v3/projects/80fd83cbc2fb4a6ab9430030122728bd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:45.111884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9048ba-c0ba-44ce-a526-3f3c04b617db None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.112386 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9048ba-c0ba-44ce-a526-3f3c04b617db None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.112516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9048ba-c0ba-44ce-a526-3f3c04b617db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.130583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-ad9048ba-c0ba-44ce-a526-3f3c04b617db None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:45.131870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ad9048ba-c0ba-44ce-a526-3f3c04b617db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:45.196557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1211/2411] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:45.204977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1e552-a129-4c82-857d-a2bac52e6e75 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.205266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1e552-a129-4c82-857d-a2bac52e6e75 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.205561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1e552-a129-4c82-857d-a2bac52e6e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.224831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-1bd1e552-a129-4c82-857d-a2bac52e6e75 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:45.226056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1bd1e552-a129-4c82-857d-a2bac52e6e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:45.234955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1201/2412] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:45.239403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9048ba-c0ba-44ce-a526-3f3c04b617db None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.249829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9048ba-c0ba-44ce-a526-3f3c04b617db None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.255477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.256149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.256431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.256706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.257448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:27:45.257757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.259425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.272071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1212/2413] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:27:45.276854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1e552-a129-4c82-857d-a2bac52e6e75 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.286386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1e552-a129-4c82-857d-a2bac52e6e75 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.291521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.292177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.292458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.292731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.293436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.295357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.316737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:45.324345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1202/2414] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:45 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:27:45.328607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.339272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85866feb-3875-4aab-8b6a-c234fd5288ea admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.399023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.399674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.399955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.400232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.401003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.403082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.424521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1213/2415] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:45 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:45.428810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.439420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0e143-67d0-4e2c-b8bf-c209c1a53e5e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.444208 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.444870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.445151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.445421 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.446140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.447977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.458563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1203/2416] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:27:45.462748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.473254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f9c9-4861-4e74-8077-e3b87e2c0c11 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.478272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f671d80-1cc3-430a-a069-8d2778629f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.479136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f671d80-1cc3-430a-a069-8d2778629f17 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.479447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f671d80-1cc3-430a-a069-8d2778629f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.479719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f671d80-1cc3-430a-a069-8d2778629f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.480458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f671d80-1cc3-430a-a069-8d2778629f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.482066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f671d80-1cc3-430a-a069-8d2778629f17 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.492985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1214/2417] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:45.497249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.507668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1f1b5-7a93-4b1d-9159-065c2a19d43d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.512674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.513576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.513859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.514137 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] PATH_INFO: `/v3/projects/1113503829494e68a1a7f95e7d5f07ce/users/53bd7efb8f99401a862aff5766a6c9ea/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.530273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1113503829494e68a1a7f95e7d5f07ce, user_id=53bd7efb8f99401a862aff5766a6c9ea, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.533019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.548268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1204/2418] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:45 2022] PUT /identity/v3/projects/1113503829494e68a1a7f95e7d5f07ce/users/53bd7efb8f99401a862aff5766a6c9ea/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:45.616089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d68ba-0cad-4963-91fd-b441447558a5 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.616381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d68ba-0cad-4963-91fd-b441447558a5 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.616765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d68ba-0cad-4963-91fd-b441447558a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.648941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-015d68ba-0cad-4963-91fd-b441447558a5 None None] MFA Rules not processed for user `53bd7efb8f99401a862aff5766a6c9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:45.650167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-015d68ba-0cad-4963-91fd-b441447558a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:45.708123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:45.719621 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1215/2419] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:45.724473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.788488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94730-43ac-4cfe-875e-9408a47a57b5 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.794099 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.794778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.795057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.795330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.796080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.798029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.827144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1205/2420] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:45 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:45.831240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d68ba-0cad-4963-91fd-b441447558a5 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.842827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d68ba-0cad-4963-91fd-b441447558a5 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.911251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.911873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.912153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.912443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.913153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.915139 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.939791 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1216/2421] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:45 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:27:45.943937 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.957602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2481bd1-f5a0-454f-b03b-a06f9107318f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:45.965238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:45.965939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:45.966216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:45.966490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:45.967234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:45.968903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:45.980557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1206/2422] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:45.985197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:45.998249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf638f8d-0944-4063-a75b-68d0088c88e8 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.003960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28850caa-9695-406c-b211-c5920cd2ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.004726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28850caa-9695-406c-b211-c5920cd2ae04 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.005007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28850caa-9695-406c-b211-c5920cd2ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.005283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28850caa-9695-406c-b211-c5920cd2ae04 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.006085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28850caa-9695-406c-b211-c5920cd2ae04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.007723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28850caa-9695-406c-b211-c5920cd2ae04 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.018522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1217/2423] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:46.023015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.033715 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9a4f-d932-466d-8c94-ceb7734897ac admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.038410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.039491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.039778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.040058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] PATH_INFO: `/v3/projects/e8aca58e581546fe956ac19c18594269/users/abd49ad62cfb4b34a99f7d44f2c7dd0c/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.055783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8aca58e581546fe956ac19c18594269, user_id=abd49ad62cfb4b34a99f7d44f2c7dd0c, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.058628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.073556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1207/2424] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:46 2022] PUT /identity/v3/projects/e8aca58e581546fe956ac19c18594269/users/abd49ad62cfb4b34a99f7d44f2c7dd0c/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:46.083753 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17da28-17b2-4014-83c2-55dd525affa0 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.084047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17da28-17b2-4014-83c2-55dd525affa0 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.084402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17da28-17b2-4014-83c2-55dd525affa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.114560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-de17da28-17b2-4014-83c2-55dd525affa0 None None] MFA Rules not processed for user `abd49ad62cfb4b34a99f7d44f2c7dd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:46.116558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-de17da28-17b2-4014-83c2-55dd525affa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:46.189299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1218/2425] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:46.193738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.258647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e7e0-c3a8-484d-9c80-6e553b3d4682 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.266966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.267867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.268153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.268554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.269537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.272181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.293931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:46.305315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1208/2426] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:46 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:46.307652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17da28-17b2-4014-83c2-55dd525affa0 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.321051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17da28-17b2-4014-83c2-55dd525affa0 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.391746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.392435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.392917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.393199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.394053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.395879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.404167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:46.421382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1219/2427] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:46 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:27:46.422492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.433969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2edc22-7259-43a6-8cea-95d837da4dc9 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.438560 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.439237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.439553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.439826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.440559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.442174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.457305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1209/2428] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:46.458489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.469328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef57d1f-f68b-4fd8-aba7-719849d8e889 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.473874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.474817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.475096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.475382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] PATH_INFO: `/v3/projects/041ad80c8bf34b9aa387b8872a160cfe/users/589b7b2f72a24d22bdcd7a3e315cafa8/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.492962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=041ad80c8bf34b9aa387b8872a160cfe, user_id=589b7b2f72a24d22bdcd7a3e315cafa8, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.495885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.508957 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1220/2429] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:46 2022] PUT /identity/v3/projects/041ad80c8bf34b9aa387b8872a160cfe/users/589b7b2f72a24d22bdcd7a3e315cafa8/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:46.514165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.523036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d2e55-6d9b-4b64-a04a-a2596791821e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.581826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25737e3-cc6e-46f3-8772-c51e95a291ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.582506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25737e3-cc6e-46f3-8772-c51e95a291ff admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.582794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25737e3-cc6e-46f3-8772-c51e95a291ff admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.583078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25737e3-cc6e-46f3-8772-c51e95a291ff admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.583854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25737e3-cc6e-46f3-8772-c51e95a291ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.585475 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25737e3-cc6e-46f3-8772-c51e95a291ff admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.595780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1210/2430] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:46.600023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.611123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519e80c-641c-4dd7-b44c-116466d7dd91 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.616070 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.616953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.617246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.617521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] PATH_INFO: `/v3/projects/041ad80c8bf34b9aa387b8872a160cfe/users/589b7b2f72a24d22bdcd7a3e315cafa8/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.619847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=041ad80c8bf34b9aa387b8872a160cfe, user_id=589b7b2f72a24d22bdcd7a3e315cafa8, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.622498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.633343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1221/2431] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:46 2022] PUT /identity/v3/projects/041ad80c8bf34b9aa387b8872a160cfe/users/589b7b2f72a24d22bdcd7a3e315cafa8/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:46.643722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7292-afc1-4801-8823-d67a2e5a8cb2 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.644018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7292-afc1-4801-8823-d67a2e5a8cb2 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.644296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7292-afc1-4801-8823-d67a2e5a8cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.672708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-30cf7292-afc1-4801-8823-d67a2e5a8cb2 None None] MFA Rules not processed for user `589b7b2f72a24d22bdcd7a3e315cafa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:46.673860 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-30cf7292-afc1-4801-8823-d67a2e5a8cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:46.747452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1211/2432] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:46.759592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:46.761793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:46.821905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdd3d-3e70-46ca-ac00-5d3363d2efc6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:46.826135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:46.826815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:46.827101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:46.827399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:46.890378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:46.891878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:46.901786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1222/2433] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:48.417826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7292-afc1-4801-8823-d67a2e5a8cb2 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:48.428696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7292-afc1-4801-8823-d67a2e5a8cb2 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:48.433338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:48.434121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:48.434474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:48.434830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:48.495355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:48.496851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:48.505909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1212/2434] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:49.088111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:49.097421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c4c25-33de-4dc7-aef9-2580b81c39e8 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:49.101852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681c428-5a6e-4a1b-8643-56281e590a8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:49.102505 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681c428-5a6e-4a1b-8643-56281e590a8a service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:49.102790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681c428-5a6e-4a1b-8643-56281e590a8a service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:49.103063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681c428-5a6e-4a1b-8643-56281e590a8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:49.113834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681c428-5a6e-4a1b-8643-56281e590a8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:49.115302 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681c428-5a6e-4a1b-8643-56281e590a8a service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:49.126248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1223/2435] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:51.921284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584644cc-7328-4850-af07-2cde89380d28 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:51.921937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584644cc-7328-4850-af07-2cde89380d28 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:51.921937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584644cc-7328-4850-af07-2cde89380d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:51.926039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:51.928383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:51.941632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:51.961387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-584644cc-7328-4850-af07-2cde89380d28 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:51.962661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-584644cc-7328-4850-af07-2cde89380d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:52.006190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:52.013888 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5b625-683f-4b37-86d3-4b9866bc4c1a service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:52.017781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:52.018450 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:52.018735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:52.019011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:52.021005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:52.028711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1224/2436] 10.222.0.87 () {58 vars in 967 bytes} [Fri Jun 17 13:27:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:52.041885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584644cc-7328-4850-af07-2cde89380d28 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:52.043344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-584644cc-7328-4850-af07-2cde89380d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:52.145566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:52.147423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:52.149414 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584644cc-7328-4850-af07-2cde89380d28 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:52.158143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb019e-b7ab-43ab-bf8d-1a4b3aa0398b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:52.159148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb019e-b7ab-43ab-bf8d-1a4b3aa0398b service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:52.159628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb019e-b7ab-43ab-bf8d-1a4b3aa0398b service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:52.160056 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb019e-b7ab-43ab-bf8d-1a4b3aa0398b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:52.181558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1213/2437] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:52.186657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb019e-b7ab-43ab-bf8d-1a4b3aa0398b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:52.188501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb019e-b7ab-43ab-bf8d-1a4b3aa0398b service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:52.199805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1225/2438] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:27:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 13:27:53.031946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:53.100739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a6045-a1ba-49b5-a1db-160352b7562c service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:53.106998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:53.107929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:53.108570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:53.108840 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] PATH_INFO: `/v3/users/53bd7efb8f99401a862aff5766a6c9ea` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:53.110744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bd7efb8f99401a862aff5766a6c9ea)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:53.112761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:53.127710 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:53.160462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bd7efb8f99401a862aff5766a6c9ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.173375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bd7efb8f99401a862aff5766a6c9ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.180139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bd7efb8f99401a862aff5766a6c9ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.189264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bd7efb8f99401a862aff5766a6c9ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.189879 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bd7efb8f99401a862aff5766a6c9ea'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.196044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1214/2439] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:53 2022] DELETE /identity/v3/users/53bd7efb8f99401a862aff5766a6c9ea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:53.200825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:53.201316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:53.201421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:53.221149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:53.222301 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:53.284625 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1226/2440] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:53.295006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:53.296564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:53.359590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6007e-12e9-4a76-a4b1-475c2a6ae8d8 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:53.363532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:53.364235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:53.364516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:53.364790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:53.374923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:53.376397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:53.386331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1215/2441] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:53.738881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:53.740326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:53.752974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c844e1-99f8-4199-a1ed-7aab11bceb7e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:53.759125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:53.760122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:53.760400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:53.760706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] PATH_INFO: `/v3/projects/1113503829494e68a1a7f95e7d5f07ce` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:53.761938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1113503829494e68a1a7f95e7d5f07ce)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:53.763945 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:53.816102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1113503829494e68a1a7f95e7d5f07ce', 'request_id': 'req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.822523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1113503829494e68a1a7f95e7d5f07ce', 'request_id': 'req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.828374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1113503829494e68a1a7f95e7d5f07ce', 'request_id': 'req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.837827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:53.843298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] The token cache is being invalidate because project 1113503829494e68a1a7f95e7d5f07ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:53.843789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.845168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.846757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1227/2442] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:53 2022] DELETE /identity/v3/projects/1113503829494e68a1a7f95e7d5f07ce => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:53.850821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:53.861505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef068567-2793-4fb6-aa6d-f0997227b2f9 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:53.923319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:53.924241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:53.924525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:53.924804 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] PATH_INFO: `/v3/users/abd49ad62cfb4b34a99f7d44f2c7dd0c` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:53.925985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd49ad62cfb4b34a99f7d44f2c7dd0c)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:53.927978 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:53.966890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:53.977255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd49ad62cfb4b34a99f7d44f2c7dd0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:53.992686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd49ad62cfb4b34a99f7d44f2c7dd0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.000451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd49ad62cfb4b34a99f7d44f2c7dd0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.008488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd49ad62cfb4b34a99f7d44f2c7dd0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.009155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd49ad62cfb4b34a99f7d44f2c7dd0c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.015241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1216/2443] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:53 2022] DELETE /identity/v3/users/abd49ad62cfb4b34a99f7d44f2c7dd0c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:54.305076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:54.306608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:54.320319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec2d0-52f0-4731-ae6b-d93a786fe9e2 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:54.383064 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:54.383989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:54.384286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:54.384735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] PATH_INFO: `/v3/projects/e8aca58e581546fe956ac19c18594269` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:54.385995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8aca58e581546fe956ac19c18594269)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:54.388002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:54.437144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8aca58e581546fe956ac19c18594269', 'request_id': 'req-95c5129a-d105-4ae4-8337-97dbb0515d11'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.445159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8aca58e581546fe956ac19c18594269', 'request_id': 'req-95c5129a-d105-4ae4-8337-97dbb0515d11'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.450093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8aca58e581546fe956ac19c18594269', 'request_id': 'req-95c5129a-d105-4ae4-8337-97dbb0515d11'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.464604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] The token cache is being invalidate because project e8aca58e581546fe956ac19c18594269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:54.464874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.465528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.467135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1228/2444] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:54 2022] DELETE /identity/v3/projects/e8aca58e581546fe956ac19c18594269 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:54.471863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:54.474022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:54.489116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02841c-3f1d-4ac8-9d07-5c5f480308d4 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:54.547938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:54.548741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:54.549023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:54.549318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] PATH_INFO: `/v3/users/589b7b2f72a24d22bdcd7a3e315cafa8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:54.550523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=589b7b2f72a24d22bdcd7a3e315cafa8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:54.552524 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:54.600101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589b7b2f72a24d22bdcd7a3e315cafa8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.612268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589b7b2f72a24d22bdcd7a3e315cafa8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.619630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589b7b2f72a24d22bdcd7a3e315cafa8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.628297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589b7b2f72a24d22bdcd7a3e315cafa8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.628929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-052c900a-0bc1-4cc3-b68a-437bf497d23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589b7b2f72a24d22bdcd7a3e315cafa8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:54.635746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1217/2445] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:54 2022] DELETE /identity/v3/users/589b7b2f72a24d22bdcd7a3e315cafa8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:54.942402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:54.943933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:54.959116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5129a-d105-4ae4-8337-97dbb0515d11 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.019487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.020357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.020643 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.020922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] PATH_INFO: `/v3/projects/041ad80c8bf34b9aa387b8872a160cfe` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.022125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=041ad80c8bf34b9aa387b8872a160cfe)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.024097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.062845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:55.078735 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '041ad80c8bf34b9aa387b8872a160cfe', 'request_id': 'req-8955fe59-da80-4204-a0c1-b868dea914cf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:55.084827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '041ad80c8bf34b9aa387b8872a160cfe', 'request_id': 'req-8955fe59-da80-4204-a0c1-b868dea914cf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:55.091199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '041ad80c8bf34b9aa387b8872a160cfe', 'request_id': 'req-8955fe59-da80-4204-a0c1-b868dea914cf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:55.104646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] The token cache is being invalidate because project 041ad80c8bf34b9aa387b8872a160cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:55.104935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:55.105553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8955fe59-da80-4204-a0c1-b868dea914cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:55.107112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1229/2446] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:54 2022] DELETE /identity/v3/projects/041ad80c8bf34b9aa387b8872a160cfe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:55.113867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319cfa5-899d-4a2a-b15d-bcf92ffaed54 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.114185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319cfa5-899d-4a2a-b15d-bcf92ffaed54 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.114466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319cfa5-899d-4a2a-b15d-bcf92ffaed54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.132702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4319cfa5-899d-4a2a-b15d-bcf92ffaed54 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:55.133913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4319cfa5-899d-4a2a-b15d-bcf92ffaed54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:55.164636 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:55.197419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1218/2447] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:55.205839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6e1c9-f301-466a-8682-1ee87e5eff0a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.206130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6e1c9-f301-466a-8682-1ee87e5eff0a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.206431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6e1c9-f301-466a-8682-1ee87e5eff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.227242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-49a6e1c9-f301-466a-8682-1ee87e5eff0a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:55.228729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-49a6e1c9-f301-466a-8682-1ee87e5eff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:55.239358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1230/2448] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:55.245863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319cfa5-899d-4a2a-b15d-bcf92ffaed54 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.261052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319cfa5-899d-4a2a-b15d-bcf92ffaed54 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.267036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.267781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.268090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.268373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.269156 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:27:55.269464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.271090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.283344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1219/2449] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:27:55.288080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6e1c9-f301-466a-8682-1ee87e5eff0a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.297881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6e1c9-f301-466a-8682-1ee87e5eff0a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.303628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.304279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.304568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.304841 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.305541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.307683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.336799 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1231/2450] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:55 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:55.341591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.358326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6111f6-8589-4e54-a1e1-a995687362a5 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.421581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.422265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.422551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.422834 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.423571 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.425619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.447326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1220/2451] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:55 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:55.451796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.462924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b47b1-dee5-454f-abfb-3bcd26c8ea8b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.468268 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.468936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.469217 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.469489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.470219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.471858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.483351 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1232/2452] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:55.487556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.498538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585ada4-9cd3-4aa0-823e-f5480512c270 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.504546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a326b2b-ee3a-4074-b3f2-43769fef89b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.505225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a326b2b-ee3a-4074-b3f2-43769fef89b8 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.505513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a326b2b-ee3a-4074-b3f2-43769fef89b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.505793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a326b2b-ee3a-4074-b3f2-43769fef89b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.506518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a326b2b-ee3a-4074-b3f2-43769fef89b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.508195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a326b2b-ee3a-4074-b3f2-43769fef89b8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.523587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1221/2453] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:55.524886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.535987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f263-5985-40ab-b9cb-87a602f35fec admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.540899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.541901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.542186 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.542451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] PATH_INFO: `/v3/projects/3370bd7a860b479bb36782fc9a646290/users/8f9b220b83da4cd1bb0b331848e5c374/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.559585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3370bd7a860b479bb36782fc9a646290, user_id=8f9b220b83da4cd1bb0b331848e5c374, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.563903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.579965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1233/2454] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:55 2022] PUT /identity/v3/projects/3370bd7a860b479bb36782fc9a646290/users/8f9b220b83da4cd1bb0b331848e5c374/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:55.590959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870cf7-617e-426a-a7ef-efd443987f5e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.591186 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870cf7-617e-426a-a7ef-efd443987f5e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.591434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870cf7-617e-426a-a7ef-efd443987f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.621749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-1a870cf7-617e-426a-a7ef-efd443987f5e None None] MFA Rules not processed for user `8f9b220b83da4cd1bb0b331848e5c374`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:55.622977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-1a870cf7-617e-426a-a7ef-efd443987f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:55.707335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1222/2455] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:55.721426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.724534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:55.796266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ecb88-2d6e-4dd7-9e88-4632e88706f4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.800350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359b79-dbe3-4772-bded-1282a6a05117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.801008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359b79-dbe3-4772-bded-1282a6a05117 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.801285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359b79-dbe3-4772-bded-1282a6a05117 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.801559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359b79-dbe3-4772-bded-1282a6a05117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.810357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359b79-dbe3-4772-bded-1282a6a05117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.811839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359b79-dbe3-4772-bded-1282a6a05117 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.820768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1234/2456] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:55.845679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870cf7-617e-426a-a7ef-efd443987f5e None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:55.880838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:55.913120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870cf7-617e-426a-a7ef-efd443987f5e None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:55.918319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:55.919202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:55.919507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:55.919770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] PATH_INFO: `/v3/users/8f9b220b83da4cd1bb0b331848e5c374` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:55.920989 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f9b220b83da4cd1bb0b331848e5c374)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:55.922922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:55.984808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f9b220b83da4cd1bb0b331848e5c374'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:55.998097 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f9b220b83da4cd1bb0b331848e5c374'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.005245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f9b220b83da4cd1bb0b331848e5c374'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.014072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f9b220b83da4cd1bb0b331848e5c374'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.014714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f9b220b83da4cd1bb0b331848e5c374'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.020813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1223/2457] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:27:55 2022] DELETE /identity/v3/users/8f9b220b83da4cd1bb0b331848e5c374 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:56.025829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.026121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.026400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.044781 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:56.046018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:56.104515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:56.120823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1235/2458] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:56.127119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:56.128669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:56.193085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5face8a-1cd0-4be0-9052-f0b2d270f7e2 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:56.198247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e9bcd-b628-433b-8228-357424a6f99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:56.198899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9bcd-b628-433b-8228-357424a6f99a service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.199180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9bcd-b628-433b-8228-357424a6f99a service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.199486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9bcd-b628-433b-8228-357424a6f99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.210424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e9bcd-b628-433b-8228-357424a6f99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:56.211908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e9bcd-b628-433b-8228-357424a6f99a service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:56.222321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1224/2459] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:56.522656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:56.524222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:56.536342 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b3851-6bc2-4476-a31f-285cc55886e9 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:56.540992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:56.542093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.542228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.542506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] PATH_INFO: `/v3/projects/3370bd7a860b479bb36782fc9a646290` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.543805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3370bd7a860b479bb36782fc9a646290)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:56.545743 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:56.595865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3370bd7a860b479bb36782fc9a646290', 'request_id': 'req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.602051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3370bd7a860b479bb36782fc9a646290', 'request_id': 'req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.608587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3370bd7a860b479bb36782fc9a646290', 'request_id': 'req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.621926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] The token cache is being invalidate because project 3370bd7a860b479bb36782fc9a646290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:27:56.622148 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.622773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-70d77fb4-0f86-4f81-ad60-13a4fb71dffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:27:56.624369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1236/2460] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:27:56 2022] DELETE /identity/v3/projects/3370bd7a860b479bb36782fc9a646290 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:56.629544 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753ad31-465f-4c0b-9f66-a42571f97560 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.630023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753ad31-465f-4c0b-9f66-a42571f97560 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.630148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753ad31-465f-4c0b-9f66-a42571f97560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.648213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b753ad31-465f-4c0b-9f66-a42571f97560 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:56.649415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b753ad31-465f-4c0b-9f66-a42571f97560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:56.698133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:27:56.714084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1225/2461] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:56.723020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ea682-389f-490c-9124-a2e6dd0bf125 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.723316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ea682-389f-490c-9124-a2e6dd0bf125 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.723633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ea682-389f-490c-9124-a2e6dd0bf125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.743887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-612ea682-389f-490c-9124-a2e6dd0bf125 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:56.745144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-612ea682-389f-490c-9124-a2e6dd0bf125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:56.755851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1237/2462] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:56.760722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753ad31-465f-4c0b-9f66-a42571f97560 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:56.770622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753ad31-465f-4c0b-9f66-a42571f97560 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:56.776300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:56.777002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.777315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.777566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.778306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:27:56.778617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:56.780279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:56.792193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1226/2463] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 13:27:56.796720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ea682-389f-490c-9124-a2e6dd0bf125 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:56.807003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ea682-389f-490c-9124-a2e6dd0bf125 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:56.813158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:56.813842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.814121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.814404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.815137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:56.817173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:56.846042 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1238/2464] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:27:56 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:56.850343 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:56.861843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35adc5-57f7-4096-a5b7-25b436a9ed87 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:56.924187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:56.924869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.925155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.925435 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.926142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:56.928134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:56.951755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1227/2465] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:27:56 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:27:56.957241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:56.970798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0903d0d-b827-41d8-a275-aa042e875943 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:56.976258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:56.976920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:56.977202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:56.977479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:56.978214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:56.980069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:56.990988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1239/2466] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:56.995282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:57.007018 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dcad2-7d6e-4d0f-8cb1-ab359496c863 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:57.014580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86406967-991e-4546-bb65-04d707978a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:57.015597 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86406967-991e-4546-bb65-04d707978a48 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:57.015949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86406967-991e-4546-bb65-04d707978a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:57.016284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86406967-991e-4546-bb65-04d707978a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:57.017415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86406967-991e-4546-bb65-04d707978a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:57.019960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86406967-991e-4546-bb65-04d707978a48 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:57.032678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1228/2467] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:27:57.037085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:57.048475 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754db04-4d49-46e3-8dd1-6ce79197a971 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:57.054447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:57.055334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:57.055717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:57.055991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] PATH_INFO: `/v3/projects/6fe54391d8ee4b8db37c479dd0c66757/users/344a774605764d2b8578e64a090f3506/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:57.071943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe54391d8ee4b8db37c479dd0c66757, user_id=344a774605764d2b8578e64a090f3506, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:57.075509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:57.091897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1240/2468] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:27:57 2022] PUT /identity/v3/projects/6fe54391d8ee4b8db37c479dd0c66757/users/344a774605764d2b8578e64a090f3506/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:27:57.102424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a1d54-0ef0-41da-8050-d9f3aad13d0a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:57.102806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a1d54-0ef0-41da-8050-d9f3aad13d0a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:57.103121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a1d54-0ef0-41da-8050-d9f3aad13d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:57.132241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a66a1d54-0ef0-41da-8050-d9f3aad13d0a None None] MFA Rules not processed for user `344a774605764d2b8578e64a090f3506`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:57.133480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a66a1d54-0ef0-41da-8050-d9f3aad13d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:57.199722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1229/2469] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:27:57.210872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:27:57.213006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:57.277603 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72016b92-debe-44de-8e0f-e6cab5c03ef4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:27:57.281394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:27:57.282048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:57.282324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:57.282592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:57.292279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:27:57.293727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:27:57.303184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1241/2470] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:27:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:27:58.680964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f0bf-c01f-4259-9e60-b1c26e762ef1 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:27:58.681541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f0bf-c01f-4259-9e60-b1c26e762ef1 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:27:58.681541 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f0bf-c01f-4259-9e60-b1c26e762ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:27:58.701912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b3c6f0bf-c01f-4259-9e60-b1c26e762ef1 None None] MFA Rules not processed for user `344a774605764d2b8578e64a090f3506`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:27:58.703115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b3c6f0bf-c01f-4259-9e60-b1c26e762ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:27:58.711074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1230/2471] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:27:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:03.454274 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:03.455966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:03.482209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:03.532906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66472f9-0a94-4732-b482-b2defa4ed414 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:03.537060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:03.537716 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:03.538007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:03.538283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:03.540280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:03.550209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:03.551740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:03.559852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1242/2472] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:08.608916 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f0bf-c01f-4259-9e60-b1c26e762ef1 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:08.618787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f0bf-c01f-4259-9e60-b1c26e762ef1 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:08.623241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:08.623949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:08.624228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:08.624501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:08.626635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:08.705935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:08.734589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:08.736437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:08.748413 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1231/2473] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:09.284600 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:09.286131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:09.349697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:09.351295 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:09.365666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b043f5-610b-4148-a921-465360e6385d service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:09.370773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:09.371999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:09.372373 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:09.372723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] PATH_INFO: `/v3/users/31af58bc06a14e3ba8b709b7deb75341` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:09.373987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cb1e0-0580-492a-ae83-61525ca86d55 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:09.374467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31af58bc06a14e3ba8b709b7deb75341)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:09.376641 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:09.380638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:09.381304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:09.381580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:09.381868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:09.384561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:09.397513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:09.399186 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:09.410725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1243/2474] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:09.425307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31af58bc06a14e3ba8b709b7deb75341'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.438194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31af58bc06a14e3ba8b709b7deb75341'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.445565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31af58bc06a14e3ba8b709b7deb75341'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.453403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31af58bc06a14e3ba8b709b7deb75341'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.454105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-afd6b37c-b95d-4627-a7f9-84ea8705f7b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31af58bc06a14e3ba8b709b7deb75341'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.460073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1232/2475] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:09 2022] DELETE /identity/v3/users/31af58bc06a14e3ba8b709b7deb75341 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:09.640254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:09.651739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768efae-a518-4d6a-8b6a-e7cf1974eb93 service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:09.708838 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:09.709710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:09.709989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:09.710260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] PATH_INFO: `/v3/projects/91f9f62d475d4663ae7c211c8adb78e8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:09.711502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f9f62d475d4663ae7c211c8adb78e8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:09.713430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:09.757357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:09.761207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f9f62d475d4663ae7c211c8adb78e8', 'request_id': 'req-47713da8-9063-4053-9ed2-f819fa1cb866'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.768587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f9f62d475d4663ae7c211c8adb78e8', 'request_id': 'req-47713da8-9063-4053-9ed2-f819fa1cb866'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.774096 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f9f62d475d4663ae7c211c8adb78e8', 'request_id': 'req-47713da8-9063-4053-9ed2-f819fa1cb866'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.786905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] The token cache is being invalidate because project 91f9f62d475d4663ae7c211c8adb78e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:09.787195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.787842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-47713da8-9063-4053-9ed2-f819fa1cb866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:09.789391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1244/2476] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:09 2022] DELETE /identity/v3/projects/91f9f62d475d4663ae7c211c8adb78e8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:09.795077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33190f7-dba7-449e-b0cd-cfc7660c6164 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:09.795633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33190f7-dba7-449e-b0cd-cfc7660c6164 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:09.795633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33190f7-dba7-449e-b0cd-cfc7660c6164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:09.813817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-a33190f7-dba7-449e-b0cd-cfc7660c6164 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:09.815026 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a33190f7-dba7-449e-b0cd-cfc7660c6164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:09.879601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1233/2477] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:09.887870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeab868-71a5-45d8-be1d-f44d7298eaf2 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:09.888202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeab868-71a5-45d8-be1d-f44d7298eaf2 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:09.888457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeab868-71a5-45d8-be1d-f44d7298eaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:09.907814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-dbeab868-71a5-45d8-be1d-f44d7298eaf2 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:09.909023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-dbeab868-71a5-45d8-be1d-f44d7298eaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:09.920805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1245/2478] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:09.926439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33190f7-dba7-449e-b0cd-cfc7660c6164 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:09.937611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33190f7-dba7-449e-b0cd-cfc7660c6164 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:09.942790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:09.943490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:09.943770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:09.944047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:09.944777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:09.945085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:09.946716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:09.958997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1234/2479] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:09.963675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeab868-71a5-45d8-be1d-f44d7298eaf2 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:09.973891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeab868-71a5-45d8-be1d-f44d7298eaf2 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:09.979691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:09.980349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:09.980632 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:09.980902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:09.981789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:09.984125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:10.012708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1246/2480] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:09 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:10.017063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:10.027561 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a1dc8-037f-4094-b899-1d1e991004b8 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:10.064172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:10.097056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:10.097770 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:10.098055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:10.098337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:10.099037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:10.101027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:10.127927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1235/2481] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:10 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:10.132265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:10.144523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a90d0a-23c2-4073-900e-7474aab33e25 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:10.150155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:10.150833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:10.151142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:10.151455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:10.152195 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:10.153869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:10.166071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1247/2482] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:10.170859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:10.181545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd411d-a9f0-444a-8f07-c68962accb2d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:10.186436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e57965-f43b-4140-b810-6808c7d9eb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:10.187133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57965-f43b-4140-b810-6808c7d9eb5d admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:10.187451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57965-f43b-4140-b810-6808c7d9eb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:10.187740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57965-f43b-4140-b810-6808c7d9eb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:10.188506 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e57965-f43b-4140-b810-6808c7d9eb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:10.190195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e57965-f43b-4140-b810-6808c7d9eb5d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:10.202389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1236/2483] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:10.206911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:10.218405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f06c0-146f-4c2e-9f32-0f32a779e964 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:10.223449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:10.224376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:10.224660 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:10.224937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] PATH_INFO: `/v3/projects/bf354ea8d6114c5193da9f445bfdc8bc/users/4bd49959eabb482ba67b68a2f581d514/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:10.240721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf354ea8d6114c5193da9f445bfdc8bc, user_id=4bd49959eabb482ba67b68a2f581d514, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:10.243421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:10.256983 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1248/2484] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:10 2022] PUT /identity/v3/projects/bf354ea8d6114c5193da9f445bfdc8bc/users/4bd49959eabb482ba67b68a2f581d514/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:10.267362 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f8cb7-567d-411c-8420-e8c095aefca7 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:10.267681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f8cb7-567d-411c-8420-e8c095aefca7 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:10.268005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f8cb7-567d-411c-8420-e8c095aefca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:10.294822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8e0f8cb7-567d-411c-8420-e8c095aefca7 None None] MFA Rules not processed for user `4bd49959eabb482ba67b68a2f581d514`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:10.296065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8e0f8cb7-567d-411c-8420-e8c095aefca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:10.362452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1237/2485] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:10.374653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:10.376837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:10.444388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15bd6-1982-480f-8705-b0c21c27665f admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:10.448736 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:10.449407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:10.449688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:10.449977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:10.461334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:10.462786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:10.472942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1249/2486] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:14.776269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813ff5-d006-4faa-862d-50e22fb3e1bd None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:14.776764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813ff5-d006-4faa-862d-50e22fb3e1bd None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:14.776885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813ff5-d006-4faa-862d-50e22fb3e1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:14.796206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-10813ff5-d006-4faa-862d-50e22fb3e1bd None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:14.797388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-10813ff5-d006-4faa-862d-50e22fb3e1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:14.855241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1238/2487] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 13:28:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:14.877675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:14.879363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:14.960383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660ae4-45b5-4e9a-b0b9-078edc1530f8 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:14.965931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:14.966606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:14.966878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:14.967157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:14.975986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:14.977611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:14.983076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:14.987395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1250/2488] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:28:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:28:15.697262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bdbdc-2f7f-4c4b-8b04-77d4d8481834 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:15.697668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bdbdc-2f7f-4c4b-8b04-77d4d8481834 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:15.698014 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bdbdc-2f7f-4c4b-8b04-77d4d8481834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:15.720808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-bf4bdbdc-2f7f-4c4b-8b04-77d4d8481834 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:15.722129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bf4bdbdc-2f7f-4c4b-8b04-77d4d8481834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:15.730043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1239/2489] 10.222.0.87 () {58 vars in 967 bytes} [Fri Jun 17 13:28:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:15.743114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:15.754379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e72da7-928e-4808-98c2-8deeab1f5cc3 service swift] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:15.760889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:15.761544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:15.761827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:15.762103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:15.770924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:15.772564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:15.781501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1251/2490] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:28:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:28:15.976374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb673212-3db4-48a1-9900-37f74460223d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:15.976901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb673212-3db4-48a1-9900-37f74460223d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:15.977029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb673212-3db4-48a1-9900-37f74460223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:15.996147 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-eb673212-3db4-48a1-9900-37f74460223d None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:15.997527 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-eb673212-3db4-48a1-9900-37f74460223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:16.005233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1240/2491] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 13:28:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:16.017595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:16.029747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b61ea-6c1f-45ec-9226-5c0dbe98f2bc service swift] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:16.035746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:16.036541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:16.036853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:16.037175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:16.046626 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:16.048279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:16.057543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1252/2492] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:28:16.402404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec02c-0621-4832-9ff8-7e5e8dcdc42a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:16.402711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec02c-0621-4832-9ff8-7e5e8dcdc42a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:16.402971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec02c-0621-4832-9ff8-7e5e8dcdc42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:16.421739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c23ec02c-0621-4832-9ff8-7e5e8dcdc42a None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:16.422941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c23ec02c-0621-4832-9ff8-7e5e8dcdc42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:16.430505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1241/2493] 10.222.0.87 () {58 vars in 967 bytes} [Fri Jun 17 13:28:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:16.443098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:16.454629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219fadc-0fb3-435b-872a-cb028ffcfa89 service swift] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:16.461763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31502bc7-e50d-4d68-a77f-b694a0698782 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:16.462623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31502bc7-e50d-4d68-a77f-b694a0698782 service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:16.462978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31502bc7-e50d-4d68-a77f-b694a0698782 service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:16.463325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31502bc7-e50d-4d68-a77f-b694a0698782 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:16.476702 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31502bc7-e50d-4d68-a77f-b694a0698782 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:16.479050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31502bc7-e50d-4d68-a77f-b694a0698782 service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:16.488774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1253/2494] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:28:17.891660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ec02c-0621-4832-9ff8-7e5e8dcdc42a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:17.921452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:17.955609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ec02c-0621-4832-9ff8-7e5e8dcdc42a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:17.960190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:17.961081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:17.961374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:17.961643 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] PATH_INFO: `/v3/users/4bd49959eabb482ba67b68a2f581d514` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:17.962855 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd49959eabb482ba67b68a2f581d514)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:17.964862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:18.020377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd49959eabb482ba67b68a2f581d514'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.033967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd49959eabb482ba67b68a2f581d514'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.042217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd49959eabb482ba67b68a2f581d514'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.051320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd49959eabb482ba67b68a2f581d514'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.052052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd49959eabb482ba67b68a2f581d514'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.058048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1242/2495] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:17 2022] DELETE /identity/v3/users/4bd49959eabb482ba67b68a2f581d514 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:18.062853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:18.063420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:18.063420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:18.082128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:18.083338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:18.148585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1254/2496] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:18.158529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:18.160054 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:18.221944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512718c-a92d-41ce-b73f-e04c14d442cc admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:18.225713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:18.226397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:18.226688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:18.226970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:18.238397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:18.240794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:18.252585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1243/2497] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:18.544797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:18.546369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:18.561666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f696e35-a38a-4062-9f81-de6fd803e422 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:18.568903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:18.570069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:18.570376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:18.570531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] PATH_INFO: `/v3/projects/bf354ea8d6114c5193da9f445bfdc8bc` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:18.572320 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf354ea8d6114c5193da9f445bfdc8bc)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:18.574395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:18.625946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:18.627602 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:18.627764 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf354ea8d6114c5193da9f445bfdc8bc', 'request_id': 'req-853c632d-15d9-4176-b806-b26de8355825'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.634569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf354ea8d6114c5193da9f445bfdc8bc', 'request_id': 'req-853c632d-15d9-4176-b806-b26de8355825'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.640905 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13e334-6de1-4eef-8ad7-8c9b82b23d40 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:18.641759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf354ea8d6114c5193da9f445bfdc8bc', 'request_id': 'req-853c632d-15d9-4176-b806-b26de8355825'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.659452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] The token cache is being invalidate because project bf354ea8d6114c5193da9f445bfdc8bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:18.659805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.660401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-853c632d-15d9-4176-b806-b26de8355825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.661972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1255/2498] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:18 2022] DELETE /identity/v3/projects/bf354ea8d6114c5193da9f445bfdc8bc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:18.699832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:18.717192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:18.718107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:18.718388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:18.718674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] PATH_INFO: `/v3/users/344a774605764d2b8578e64a090f3506` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:18.719932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=344a774605764d2b8578e64a090f3506)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:18.721941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:18.781983 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '344a774605764d2b8578e64a090f3506'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.800823 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '344a774605764d2b8578e64a090f3506'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.809166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '344a774605764d2b8578e64a090f3506'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.818667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '344a774605764d2b8578e64a090f3506'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.819324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '344a774605764d2b8578e64a090f3506'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:18.826593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1244/2499] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:18 2022] DELETE /identity/v3/users/344a774605764d2b8578e64a090f3506 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:18.831593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:18.832202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:18.832478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:18.852358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:18.853758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:18.897671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:18.919979 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1256/2500] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:18.929986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:18.931837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:18.993172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96c64d-705b-4177-8875-7e8cf899e865 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:18.998311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:18.999089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:18.999398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:18.999732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:19.009961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:19.011558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:19.021816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1245/2501] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:19.208858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:19.210846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:19.224029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229df30-8c2c-4a0b-823b-3e90a72b3f21 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:19.229751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:19.230657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:19.230915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:19.231551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] PATH_INFO: `/v3/projects/6fe54391d8ee4b8db37c479dd0c66757` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:19.233828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe54391d8ee4b8db37c479dd0c66757)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:19.235846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:19.289078 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe54391d8ee4b8db37c479dd0c66757', 'request_id': 'req-d234008c-04ae-4c06-9e76-55f34d8a4633'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:19.295596 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe54391d8ee4b8db37c479dd0c66757', 'request_id': 'req-d234008c-04ae-4c06-9e76-55f34d8a4633'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:19.301561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe54391d8ee4b8db37c479dd0c66757', 'request_id': 'req-d234008c-04ae-4c06-9e76-55f34d8a4633'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:19.315608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] The token cache is being invalidate because project 6fe54391d8ee4b8db37c479dd0c66757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:19.315909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:19.316717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:19.318280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1257/2502] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:19 2022] DELETE /identity/v3/projects/6fe54391d8ee4b8db37c479dd0c66757 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:29.017080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:29.312673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:34.360894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:34.373857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a64a8-be6e-474b-9ae9-a491360577f8 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:34.441845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:34.442564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:34.442848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:34.443762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:34.446562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:34.551470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:34.553028 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:34.573121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1246/2503] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:38.545075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:38.546614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:38.566955 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234008c-04ae-4c06-9e76-55f34d8a4633 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:38.577000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:38.577908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:38.578187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:38.578460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] PATH_INFO: `/v3/users/9863e406790b413faf022554ba4b8dfe` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:38.580447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9863e406790b413faf022554ba4b8dfe)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:38.582849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:38.689208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9863e406790b413faf022554ba4b8dfe'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:38.707313 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9863e406790b413faf022554ba4b8dfe'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:38.719161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9863e406790b413faf022554ba4b8dfe'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:38.729267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9863e406790b413faf022554ba4b8dfe'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:38.730005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-b7c22f84-f487-4301-9b4e-a954cac1c532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9863e406790b413faf022554ba4b8dfe'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:38.737814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1258/2504] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:38 2022] DELETE /identity/v3/users/9863e406790b413faf022554ba4b8dfe => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:38.941912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:38.959711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628dd11-c7c5-40c4-bdb9-d9d959a68061 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.034657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.035574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.035857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.036172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] PATH_INFO: `/v3/projects/ea1bbb589db14084aff24be5d6e118a3` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.037427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea1bbb589db14084aff24be5d6e118a3)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.039282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.093224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea1bbb589db14084aff24be5d6e118a3', 'request_id': 'req-a0f27112-49d9-4946-bdcb-4b82eead5fbf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:39.096168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:39.101838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea1bbb589db14084aff24be5d6e118a3', 'request_id': 'req-a0f27112-49d9-4946-bdcb-4b82eead5fbf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:39.109470 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea1bbb589db14084aff24be5d6e118a3', 'request_id': 'req-a0f27112-49d9-4946-bdcb-4b82eead5fbf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:39.144845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] The token cache is being invalidate because project ea1bbb589db14084aff24be5d6e118a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:39.145284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:39.146652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:39.148375 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1247/2505] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:38 2022] DELETE /identity/v3/projects/ea1bbb589db14084aff24be5d6e118a3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:39.160180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81c64e-33bd-4d52-b005-e0264ebca1c1 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.160457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81c64e-33bd-4d52-b005-e0264ebca1c1 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.160731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81c64e-33bd-4d52-b005-e0264ebca1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.187580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-5f81c64e-33bd-4d52-b005-e0264ebca1c1 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:39.193144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5f81c64e-33bd-4d52-b005-e0264ebca1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:39.212481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.216460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:39.280339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1259/2506] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:39.282557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a63109-86ea-4e24-860b-f3bcdc244096 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.282849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a63109-86ea-4e24-860b-f3bcdc244096 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.283480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a63109-86ea-4e24-860b-f3bcdc244096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.289132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f27112-49d9-4946-bdcb-4b82eead5fbf admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.292925 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.293589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.293897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.294356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.296416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:39.306212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-17a63109-86ea-4e24-860b-f3bcdc244096 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:39.307507 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-17a63109-86ea-4e24-860b-f3bcdc244096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:39.310666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.312348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.317132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1260/2507] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:39.321114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a63109-86ea-4e24-860b-f3bcdc244096 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.330323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a63109-86ea-4e24-860b-f3bcdc244096 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.330712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1248/2508] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:39.334968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.335679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.335964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.336240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.336984 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:39.337290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.338904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.354363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1261/2509] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 13:28:39.355534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.366036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04d17-b5b9-45f5-8f63-3290c6bcfdcc service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.370816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.371525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.371818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.372256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.372968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.375022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.404600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1249/2510] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:39 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:39.408439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.419172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620d63-660e-4f63-97d6-a44e9abd3e38 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.473530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:39.492973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.494036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.494323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.494602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.495316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.497349 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.520726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1262/2511] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:39 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:39.524749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.536209 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5e6e0-013c-4fc7-a4fc-96d1ab87af47 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.541346 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.541955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.542243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.542533 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.543281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.545054 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.556614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1250/2512] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:39.560918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.571517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b700a0-7000-4e5c-a886-e4c2f5dcd59d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.576872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b04d1a-2a4b-4bd2-ad5b-b9bc6c20943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.577555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b04d1a-2a4b-4bd2-ad5b-b9bc6c20943c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.577835 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b04d1a-2a4b-4bd2-ad5b-b9bc6c20943c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.578110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b04d1a-2a4b-4bd2-ad5b-b9bc6c20943c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.578850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b04d1a-2a4b-4bd2-ad5b-b9bc6c20943c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.581787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b04d1a-2a4b-4bd2-ad5b-b9bc6c20943c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.595720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1263/2513] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:39.597038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.608385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6221b-cefe-4031-ad13-3213f956cc3f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.613374 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.614288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.614581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.614853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] PATH_INFO: `/v3/projects/e9d05e280ff24ba8b2ae4f0bc0161e80/users/21cba6c38f174886b175f5239d18a25d/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.631284 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d05e280ff24ba8b2ae4f0bc0161e80, user_id=21cba6c38f174886b175f5239d18a25d, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.634002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.648423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1251/2514] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:39 2022] PUT /identity/v3/projects/e9d05e280ff24ba8b2ae4f0bc0161e80/users/21cba6c38f174886b175f5239d18a25d/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:39.658995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b351e2-c998-44eb-aaf7-51b04cb54bd9 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.659270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b351e2-c998-44eb-aaf7-51b04cb54bd9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.659604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b351e2-c998-44eb-aaf7-51b04cb54bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.687445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-01b351e2-c998-44eb-aaf7-51b04cb54bd9 None None] MFA Rules not processed for user `21cba6c38f174886b175f5239d18a25d`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:39.688691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-01b351e2-c998-44eb-aaf7-51b04cb54bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:39.755407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1264/2515] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:39.766903 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:39.768819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:39.846647 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d468be8-b5a6-494a-ad40-f6fd9131a545 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:39.852974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:39.853665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:39.853969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:39.854250 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:39.864599 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:39.866351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:39.876941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1252/2516] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:41.713361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b351e2-c998-44eb-aaf7-51b04cb54bd9 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:41.776720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b351e2-c998-44eb-aaf7-51b04cb54bd9 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:41.780704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:41.781446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:41.781800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:41.782146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:41.784065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:41.849448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:41.850935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:41.862387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1265/2517] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:42.630279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:42.632760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:42.657633 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b430-cd19-4b4e-bafe-0373e4c2ee4f service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:42.662519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:42.663785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:42.664068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:42.664325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] PATH_INFO: `/v3/users/f3bfb04b34ed4761bb09be7b29d1ec8f` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:42.665685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3bfb04b34ed4761bb09be7b29d1ec8f)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:42.667679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:42.673972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:42.736424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3bfb04b34ed4761bb09be7b29d1ec8f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:42.752793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3bfb04b34ed4761bb09be7b29d1ec8f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:42.759904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3bfb04b34ed4761bb09be7b29d1ec8f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:42.769041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3bfb04b34ed4761bb09be7b29d1ec8f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:42.769992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-43c365c7-f945-44b2-84b9-23d7f05a2a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3bfb04b34ed4761bb09be7b29d1ec8f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:42.777403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1253/2518] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:42 2022] DELETE /identity/v3/users/f3bfb04b34ed4761bb09be7b29d1ec8f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:42.961924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:42.973213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8d537-00f4-440d-9e07-4cc67541a64b service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:42.983540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:43.040801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.041683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.041966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.042253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] PATH_INFO: `/v3/projects/ead9b99d062e49a88a88e8b37a3b3519` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.043479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead9b99d062e49a88a88e8b37a3b3519)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.045928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.095460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead9b99d062e49a88a88e8b37a3b3519', 'request_id': 'req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:43.102149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead9b99d062e49a88a88e8b37a3b3519', 'request_id': 'req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:43.108362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead9b99d062e49a88a88e8b37a3b3519', 'request_id': 'req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:43.131231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] The token cache is being invalidate because project ead9b99d062e49a88a88e8b37a3b3519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:43.131561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:43.132226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4ae9f9f3-6dc7-4556-8a42-d4a0bdfb593b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:43.133881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1266/2519] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:42 2022] DELETE /identity/v3/projects/ead9b99d062e49a88a88e8b37a3b3519 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:43.139155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb80ae3-7608-427a-9bd9-ff6602a71daf None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.139476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb80ae3-7608-427a-9bd9-ff6602a71daf None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.139742 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb80ae3-7608-427a-9bd9-ff6602a71daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.157291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4eb80ae3-7608-427a-9bd9-ff6602a71daf None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:43.158505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4eb80ae3-7608-427a-9bd9-ff6602a71daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:43.226537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1254/2520] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:43.235501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e4e2a-bf26-4941-bdef-46c3569f5606 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.235801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e4e2a-bf26-4941-bdef-46c3569f5606 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.236409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e4e2a-bf26-4941-bdef-46c3569f5606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.258618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7c1e4e2a-bf26-4941-bdef-46c3569f5606 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:43.259953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7c1e4e2a-bf26-4941-bdef-46c3569f5606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:43.268656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1267/2521] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:43.273075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb80ae3-7608-427a-9bd9-ff6602a71daf None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.283364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb80ae3-7608-427a-9bd9-ff6602a71daf None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.289123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.290126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.290797 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.291089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.291940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:43.292225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.294053 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.307598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1255/2522] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:43.312698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e4e2a-bf26-4941-bdef-46c3569f5606 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.322829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e4e2a-bf26-4941-bdef-46c3569f5606 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.329377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.330079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.330363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.330642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.331455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.333592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.366479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1268/2523] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:43 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:43.370668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.383731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475e03-9550-4056-abe4-22134fa3e989 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.451611 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.452303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.452587 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.452864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.453694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.455665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.472313 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:43.485299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1256/2524] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:43 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:43.485942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.496776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc5802-01b3-4e5f-9f47-05c66ca37082 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.502420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.503101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.503350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.503850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.504727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.506336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.518560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1269/2525] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:43.522663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.533792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2cafe-6d9d-464f-b3e2-4f48fb702bcf admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.538568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010fb08-2eea-4484-bd16-4e9aec3d7732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.539270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010fb08-2eea-4484-bd16-4e9aec3d7732 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.539589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010fb08-2eea-4484-bd16-4e9aec3d7732 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.539869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010fb08-2eea-4484-bd16-4e9aec3d7732 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.540606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010fb08-2eea-4484-bd16-4e9aec3d7732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.542227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010fb08-2eea-4484-bd16-4e9aec3d7732 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.553478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1257/2526] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:43.557922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.568535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72756de8-ba85-4eea-88b7-e0310c3ae647 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.573691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.574698 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.575190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.576134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] PATH_INFO: `/v3/projects/609fa780f64546e89b3c4befaae414fa/users/41cdeb3654c6427aaaaa5551e0c2f8ea/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.594462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609fa780f64546e89b3c4befaae414fa, user_id=41cdeb3654c6427aaaaa5551e0c2f8ea, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.597190 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.612771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1270/2527] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:43 2022] PUT /identity/v3/projects/609fa780f64546e89b3c4befaae414fa/users/41cdeb3654c6427aaaaa5551e0c2f8ea/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:43.624550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194eb6-108c-4332-a741-320aed81ebbb None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.624827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194eb6-108c-4332-a741-320aed81ebbb None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.625112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194eb6-108c-4332-a741-320aed81ebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.655749 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-aa194eb6-108c-4332-a741-320aed81ebbb None None] MFA Rules not processed for user `41cdeb3654c6427aaaaa5551e0c2f8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:43.657152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-aa194eb6-108c-4332-a741-320aed81ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:43.727707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1258/2528] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:43.733036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.814528 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:43.849081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622ff6-0ee1-4d6a-a556-74a7f59b0448 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.856737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.858031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.858393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.858739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.859630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:43.861742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:43.899478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1271/2529] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:43 2022] POST /identity/v3/projects => generated 403 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:43.901219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa194eb6-108c-4332-a741-320aed81ebbb None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:43.926165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa194eb6-108c-4332-a741-320aed81ebbb None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:43.996769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:43.998013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:43.998013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:43.998298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:43.999196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:44.001582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:44.025016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1259/2530] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:43 2022] POST /identity/v3/users => generated 308 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:44.029241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:44.041691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c33ae-e925-42f3-aee0-534ea8220287 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:44.047409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:44.047830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.048310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.048663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.049694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:44.051273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:44.066954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1272/2531] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:44.068232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:44.080994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe661b-29d6-45fd-89c6-eef8fdf2dd9c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:44.085814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:44.086754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.087044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.087357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] PATH_INFO: `/v3/projects/ea5a961dc7604e7cb6d81e5287b471b9/users/2fc2ed25932145fd9af4ccb2f06fee44/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.109712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5a961dc7604e7cb6d81e5287b471b9, user_id=2fc2ed25932145fd9af4ccb2f06fee44, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:44.113744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:44.135340 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1260/2532] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:44 2022] PUT /identity/v3/projects/ea5a961dc7604e7cb6d81e5287b471b9/users/2fc2ed25932145fd9af4ccb2f06fee44/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:44.139409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:44.151703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e7fbb-bb71-4228-843a-b952f05492e6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:44.226903 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbebb5-74d4-49b3-b40a-2208cc711e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:44.228107 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbebb5-74d4-49b3-b40a-2208cc711e3c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.228616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbebb5-74d4-49b3-b40a-2208cc711e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.229276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbebb5-74d4-49b3-b40a-2208cc711e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.230343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbebb5-74d4-49b3-b40a-2208cc711e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:44.234400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbebb5-74d4-49b3-b40a-2208cc711e3c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:44.246770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1273/2533] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:44.251068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:44.265859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef5ea9-c2fa-4524-8123-44febfc91f6d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:44.271528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:44.272678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.272961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.273274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] PATH_INFO: `/v3/projects/ea5a961dc7604e7cb6d81e5287b471b9/users/2fc2ed25932145fd9af4ccb2f06fee44/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.276288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5a961dc7604e7cb6d81e5287b471b9, user_id=2fc2ed25932145fd9af4ccb2f06fee44, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:44.281877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:44.303444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1261/2534] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:44 2022] PUT /identity/v3/projects/ea5a961dc7604e7cb6d81e5287b471b9/users/2fc2ed25932145fd9af4ccb2f06fee44/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:44.307261 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c783-53b7-4c8a-9ad8-e77c2890d5e2 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.307564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c783-53b7-4c8a-9ad8-e77c2890d5e2 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.307839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c783-53b7-4c8a-9ad8-e77c2890d5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.345199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-cb37c783-53b7-4c8a-9ad8-e77c2890d5e2 None None] MFA Rules not processed for user `2fc2ed25932145fd9af4ccb2f06fee44`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:44.346571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-cb37c783-53b7-4c8a-9ad8-e77c2890d5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:44.420413 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1274/2535] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:44.431623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:44.433703 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:44.471919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:44.496327 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32a55-22b5-4bbe-8209-6c863f51232f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:44.500314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053ff9c-a0b0-44cd-af71-815718e62159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:44.501024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053ff9c-a0b0-44cd-af71-815718e62159 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.501291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053ff9c-a0b0-44cd-af71-815718e62159 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.501570 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053ff9c-a0b0-44cd-af71-815718e62159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.512268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053ff9c-a0b0-44cd-af71-815718e62159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:44.513796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053ff9c-a0b0-44cd-af71-815718e62159 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:44.525127 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1262/2536] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:44.676367 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c783-53b7-4c8a-9ad8-e77c2890d5e2 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:44.720755 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:44.778392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c783-53b7-4c8a-9ad8-e77c2890d5e2 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:44.785996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:44.787088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.787502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.787891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] PATH_INFO: `/v3/users/41cdeb3654c6427aaaaa5551e0c2f8ea` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.789241 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41cdeb3654c6427aaaaa5551e0c2f8ea)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:44.791335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:44.847587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41cdeb3654c6427aaaaa5551e0c2f8ea'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:44.860653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41cdeb3654c6427aaaaa5551e0c2f8ea'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:44.868700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41cdeb3654c6427aaaaa5551e0c2f8ea'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:44.884426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41cdeb3654c6427aaaaa5551e0c2f8ea'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:44.885615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41cdeb3654c6427aaaaa5551e0c2f8ea'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:44.893897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1275/2537] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:44 2022] DELETE /identity/v3/users/41cdeb3654c6427aaaaa5551e0c2f8ea => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:44.898249 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:44.898766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:44.898899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:44.917117 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:44.918341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:45.009140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1263/2538] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:45.016366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:45.018616 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:45.086887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea45f8b-8f16-43e0-8b9d-d60d49fd9741 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:45.092260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:45.092999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:45.093287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:45.093617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:45.105182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:45.107053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:45.161024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1276/2539] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:45.639415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:45.640917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:45.655625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82ecd4-0659-45e0-b886-ead3c458f5f9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:45.660833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:45.661887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:45.662322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:45.662679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] PATH_INFO: `/v3/users/a14c1230233e43789087e9239ffb91e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:45.663946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a14c1230233e43789087e9239ffb91e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:45.666009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:45.676154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:45.677567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:45.693544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c25b97-9cb8-48fb-846a-426345e89fa7 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:45.700542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:45.701494 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:45.701802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:45.702086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] PATH_INFO: `/v3/projects/609fa780f64546e89b3c4befaae414fa` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:45.703432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=609fa780f64546e89b3c4befaae414fa)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:45.705247 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:45.733259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a14c1230233e43789087e9239ffb91e2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.746368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a14c1230233e43789087e9239ffb91e2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.753652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a14c1230233e43789087e9239ffb91e2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.757786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609fa780f64546e89b3c4befaae414fa', 'request_id': 'req-873044a1-47be-4aed-9bc3-2dab863a76ca'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.763835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a14c1230233e43789087e9239ffb91e2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.765266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a14c1230233e43789087e9239ffb91e2'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.765464 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:45.766767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609fa780f64546e89b3c4befaae414fa', 'request_id': 'req-873044a1-47be-4aed-9bc3-2dab863a76ca'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.777434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1264/2540] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:45 2022] DELETE /identity/v3/users/a14c1230233e43789087e9239ffb91e2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:45.777661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609fa780f64546e89b3c4befaae414fa', 'request_id': 'req-873044a1-47be-4aed-9bc3-2dab863a76ca'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.797890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] The token cache is being invalidate because project 609fa780f64546e89b3c4befaae414fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:45.798168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.798913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.800811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1277/2541] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:45 2022] DELETE /identity/v3/projects/609fa780f64546e89b3c4befaae414fa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:45.807358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:45.808837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:45.862376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:45.884325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bfdaa-3e60-457d-9e08-9f89dec86167 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:45.889401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:45.890312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:45.890598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:45.890870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] PATH_INFO: `/v3/users/2fc2ed25932145fd9af4ccb2f06fee44` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:45.892125 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc2ed25932145fd9af4ccb2f06fee44)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:45.894094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:45.941397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc2ed25932145fd9af4ccb2f06fee44'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.955252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc2ed25932145fd9af4ccb2f06fee44'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.962530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc2ed25932145fd9af4ccb2f06fee44'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.969653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:45.971161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:45.971938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc2ed25932145fd9af4ccb2f06fee44'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.972961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-a9fdda64-3604-4abd-b9a9-643fc38cd8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc2ed25932145fd9af4ccb2f06fee44'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:45.981819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1265/2542] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:45 2022] DELETE /identity/v3/users/2fc2ed25932145fd9af4ccb2f06fee44 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:46.052034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873044a1-47be-4aed-9bc3-2dab863a76ca admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.114665 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.115618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.115905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.116182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] PATH_INFO: `/v3/projects/b3495f3daaa44dc9b20a08f3e1128bfa` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.117671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3495f3daaa44dc9b20a08f3e1128bfa)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.119355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.171142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3495f3daaa44dc9b20a08f3e1128bfa', 'request_id': 'req-6ae63a97-579f-4303-b3ff-d065c4f44f07'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.178370 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3495f3daaa44dc9b20a08f3e1128bfa', 'request_id': 'req-6ae63a97-579f-4303-b3ff-d065c4f44f07'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.184003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3495f3daaa44dc9b20a08f3e1128bfa', 'request_id': 'req-6ae63a97-579f-4303-b3ff-d065c4f44f07'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.197654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] The token cache is being invalidate because project b3495f3daaa44dc9b20a08f3e1128bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:46.198004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.198699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.200587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1278/2543] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:45 2022] DELETE /identity/v3/projects/b3495f3daaa44dc9b20a08f3e1128bfa => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:46.206077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723d1a2-e0fc-4910-a01c-22de978bd4d5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.206376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723d1a2-e0fc-4910-a01c-22de978bd4d5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.206658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723d1a2-e0fc-4910-a01c-22de978bd4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.226755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b723d1a2-e0fc-4910-a01c-22de978bd4d5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:46.228157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b723d1a2-e0fc-4910-a01c-22de978bd4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.317449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.318986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.323481 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1266/2544] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:46.325207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.325652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.325944 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.337522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63a97-579f-4303-b3ff-d065c4f44f07 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.344678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.345919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.347792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.348097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] PATH_INFO: `/v3/projects/ea5a961dc7604e7cb6d81e5287b471b9` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.349682 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea5a961dc7604e7cb6d81e5287b471b9)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.349808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:46.350941 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.354446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.364566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1267/2545] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:46.364566 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.366480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.367047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:46.382140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883447f-a163-4c7c-89e9-f8bdcbe32a3a None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.388985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.389869 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.390157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.390431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] PATH_INFO: `/v3/users/21cba6c38f174886b175f5239d18a25d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.391886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21cba6c38f174886b175f5239d18a25d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.394485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.416846 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea5a961dc7604e7cb6d81e5287b471b9', 'request_id': 'req-8199c3dc-f533-4839-ab4c-dd5a27fdac14'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.424672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea5a961dc7604e7cb6d81e5287b471b9', 'request_id': 'req-8199c3dc-f533-4839-ab4c-dd5a27fdac14'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.431815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea5a961dc7604e7cb6d81e5287b471b9', 'request_id': 'req-8199c3dc-f533-4839-ab4c-dd5a27fdac14'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.454644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21cba6c38f174886b175f5239d18a25d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.457272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] The token cache is being invalidate because project ea5a961dc7604e7cb6d81e5287b471b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:46.457570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.458463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.460511 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1279/2546] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:46 2022] DELETE /identity/v3/projects/ea5a961dc7604e7cb6d81e5287b471b9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:46.460877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:46.461022 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.462800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.472210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21cba6c38f174886b175f5239d18a25d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.483149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21cba6c38f174886b175f5239d18a25d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.494264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21cba6c38f174886b175f5239d18a25d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.495047 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-4401d915-f2a0-4cf8-af4b-0c12ae3c9dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21cba6c38f174886b175f5239d18a25d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:46.501741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1268/2547] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:46 2022] DELETE /identity/v3/users/21cba6c38f174886b175f5239d18a25d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:46.503052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d239d-4c84-4abe-b1af-9dad767e5e62 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.503337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d239d-4c84-4abe-b1af-9dad767e5e62 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.503652 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d239d-4c84-4abe-b1af-9dad767e5e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.521926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-6f0d239d-4c84-4abe-b1af-9dad767e5e62 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:46.523120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6f0d239d-4c84-4abe-b1af-9dad767e5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.530163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c3dc-f533-4839-ab4c-dd5a27fdac14 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.581684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1269/2548] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:46.582999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.583288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.583594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.590963 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0aaf6-e68a-45c3-8fa9-b6326028dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.592003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0aaf6-e68a-45c3-8fa9-b6326028dbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.592359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0aaf6-e68a-45c3-8fa9-b6326028dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.592715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0aaf6-e68a-45c3-8fa9-b6326028dbe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.593538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0aaf6-e68a-45c3-8fa9-b6326028dbe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:46.593855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0aaf6-e68a-45c3-8fa9-b6326028dbe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.595512 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0aaf6-e68a-45c3-8fa9-b6326028dbe7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.604352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:46.605569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.608018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1280/2549] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:46.609379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.609674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.609951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.613860 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1270/2550] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:46.614621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.615884 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.628091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e091bc-84c8-4391-ac82-1be77c36a2da None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.630463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:46.631722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.632781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.633444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.633765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.634039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.634741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.636697 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.640620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1281/2551] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:46.641347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.642591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:46.668173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1271/2552] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:46.669671 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.677737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:46.704385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84d96f-c53d-4d82-9837-a23e8850b37e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.734874 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142542b-5450-4b3e-8d0a-0fe4be98d0b4 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.739616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.740290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.740576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.740854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.741590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:46.741904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.743540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.754961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.758898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.759035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1272/2553] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:46.759135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.759135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.759337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.767194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473caa3-95f9-468b-90cd-44e36e07ace7 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.770492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.771940 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.772110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.772620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.772907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.773316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.774019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.775990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.783150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1282/2554] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:46.788403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.800202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1273/2555] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:46.800347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a98358-e2ae-4135-9fdc-d723aaa2ab22 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.804016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.806232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.806914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.807199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.807508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.808227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.810272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.815093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d281f-b18d-47fc-bdf5-df165490e958 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.819913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.820740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.821195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.821769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.822721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.824593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.840819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1274/2556] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:46.847825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1283/2557] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:46.848057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.849123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.857882 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac009837-f13d-49f4-a94f-cf384a656e81 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.863298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:46.923860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b651f-5d01-4fe8-b976-6019ef534f89 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.925859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.927015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.927015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.927253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.927973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.929062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.929606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.929748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.930568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.930568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.930987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.932958 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.943912 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1275/2558] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:46.945727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.956394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aafa8a-770e-488f-b8e8-1211b28a475e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.962819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1284/2559] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:46 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:46.971022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5a97b-5cca-45f5-8ea2-2ca968fd8885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.972296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a97b-5cca-45f5-8ea2-2ca968fd8885 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.972585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a97b-5cca-45f5-8ea2-2ca968fd8885 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.972868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a97b-5cca-45f5-8ea2-2ca968fd8885 admin admin] PATH_INFO: `/v3/projects/d13b1026d1dd4ef999a4086a53caccf8/users/5a67c3a46833430abf18d3d890db631b/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.973605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:46.984220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb8549-0ceb-476b-a205-c699c7fc8950 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:46.989671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:46.990090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5a97b-5cca-45f5-8ea2-2ca968fd8885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d13b1026d1dd4ef999a4086a53caccf8, user_id=5a67c3a46833430abf18d3d890db631b, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.990223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:46.990472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:46.990751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:46.991526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:46.992973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5a97b-5cca-45f5-8ea2-2ca968fd8885 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:46.993118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.004176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1285/2560] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:47.007994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.009119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1276/2561] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:46 2022] PUT /identity/v3/projects/d13b1026d1dd4ef999a4086a53caccf8/users/5a67c3a46833430abf18d3d890db631b/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:47.018936 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac0142-3c97-46da-a348-1f3255987054 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.020268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ea832-76c3-4b3a-b3f9-bc6a5aedba45 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.020552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ea832-76c3-4b3a-b3f9-bc6a5aedba45 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.020961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ea832-76c3-4b3a-b3f9-bc6a5aedba45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.050152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-da8ea832-76c3-4b3a-b3f9-bc6a5aedba45 None None] MFA Rules not processed for user `5a67c3a46833430abf18d3d890db631b`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:47.051449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-da8ea832-76c3-4b3a-b3f9-bc6a5aedba45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.069820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:47.090899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.091750 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.092085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.092322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.093171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.094951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.111551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1286/2562] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:47.113862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.116487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.130977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401927f-c7ec-496c-bf83-a6981524c818 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.136010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1277/2563] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:47.136212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.136997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.137394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.137783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] PATH_INFO: `/v3/projects/e9d05e280ff24ba8b2ae4f0bc0161e80` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.139031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d05e280ff24ba8b2ae4f0bc0161e80)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.140324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ea832-76c3-4b3a-b3f9-bc6a5aedba45 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.141043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.155649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ea832-76c3-4b3a-b3f9-bc6a5aedba45 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.161344 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.162292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.162579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.162856 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] PATH_INFO: `/v3/projects/339e6c62c08041f3a501fec502d8f22b/users/fb1d6fd3a7d64310bef274cfee0c7294/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.183821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=339e6c62c08041f3a501fec502d8f22b, user_id=fb1d6fd3a7d64310bef274cfee0c7294, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.187492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.192864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d05e280ff24ba8b2ae4f0bc0161e80', 'request_id': 'req-a30106f0-28f4-4bf2-a272-87ac0063d6a1'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:47.201649 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1278/2564] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:47 2022] PUT /identity/v3/projects/339e6c62c08041f3a501fec502d8f22b/users/fb1d6fd3a7d64310bef274cfee0c7294/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:47.202451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.212420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d05e280ff24ba8b2ae4f0bc0161e80', 'request_id': 'req-a30106f0-28f4-4bf2-a272-87ac0063d6a1'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:47.218829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d05e280ff24ba8b2ae4f0bc0161e80', 'request_id': 'req-a30106f0-28f4-4bf2-a272-87ac0063d6a1'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:47.232523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] The token cache is being invalidate because project e9d05e280ff24ba8b2ae4f0bc0161e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:47.233124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:47.233788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-a30106f0-28f4-4bf2-a272-87ac0063d6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:47.235409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1287/2565] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:47 2022] DELETE /identity/v3/projects/e9d05e280ff24ba8b2ae4f0bc0161e80 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:47.236650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.236943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.237219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.256155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:47.264817 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] MFA Rules not processed for user `fb1d6fd3a7d64310bef274cfee0c7294`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:47.266050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.272246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65d002-4b3f-4958-9341-14830bee4f3b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.277593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4aa78-c0ff-4329-9144-dc93a41f4aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.278420 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4aa78-c0ff-4329-9144-dc93a41f4aac admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.278842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4aa78-c0ff-4329-9144-dc93a41f4aac admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.279229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4aa78-c0ff-4329-9144-dc93a41f4aac admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.280002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4aa78-c0ff-4329-9144-dc93a41f4aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.281971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4aa78-c0ff-4329-9144-dc93a41f4aac admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.307287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1279/2566] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:47.308601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.308892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.309169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.328057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:47.329307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.338919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1288/2567] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:47.339780 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.341256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.355179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39279f9-6194-433b-8eb7-a3f60030b531 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.394598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1280/2568] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:47.395679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.397241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.409593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42295d-e70a-4334-b613-2e0d602a17b8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.410400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:47.414707 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.415485 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.415885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.416178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.417059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631a4de-e29d-48e2-82a1-2aee423cf9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.417219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.417740 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631a4de-e29d-48e2-82a1-2aee423cf9db admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.418346 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631a4de-e29d-48e2-82a1-2aee423cf9db admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.419244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.419408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631a4de-e29d-48e2-82a1-2aee423cf9db admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.420385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631a4de-e29d-48e2-82a1-2aee423cf9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.422790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631a4de-e29d-48e2-82a1-2aee423cf9db admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.446973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1289/2569] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:47.446973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6b828-b14a-49e3-8c4e-904a60b5d009 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.446973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6b828-b14a-49e3-8c4e-904a60b5d009 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.446973 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6b828-b14a-49e3-8c4e-904a60b5d009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.453383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1281/2570] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:47.453383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.466244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-3ce6b828-b14a-49e3-8c4e-904a60b5d009 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:47.467495 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3ce6b828-b14a-49e3-8c4e-904a60b5d009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.515385 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe7c9d-f4de-4f0b-affe-e55b50d81103 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.521291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.521981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.522268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.522548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.523290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.524984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.527433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1290/2571] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:47.528389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6b828-b14a-49e3-8c4e-904a60b5d009 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.537289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1282/2572] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:47.538083 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.539441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6b828-b14a-49e3-8c4e-904a60b5d009 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.544551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.545421 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.545683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.545959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.546669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.548184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e076fe4-c624-4e4e-9121-95ebe315c20f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.548795 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.552815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.553466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.553753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.554031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.554765 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:47.555079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.556696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.570433 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1283/2573] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:47.570692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1291/2574] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:47.571248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.574831 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.581248 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b941b-2933-4325-9965-adb6472ba335 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.586116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.586316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f2ae5-bc18-4b02-a7f7-3b39615a596d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.587011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.587299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.587603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] PATH_INFO: `/v3/projects/731a22532b9f4f84ba448705cec8f96c/users/8a1ed29cc0e044288bdb2f63e05da2e5/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.591531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.592206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.592542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.592824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.593524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.595462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.604205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] RBAC: Authorizing `identity:create_grant(project_id=731a22532b9f4f84ba448705cec8f96c, user_id=8a1ed29cc0e044288bdb2f63e05da2e5, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.607621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.622425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1284/2575] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:47 2022] PUT /identity/v3/projects/731a22532b9f4f84ba448705cec8f96c/users/8a1ed29cc0e044288bdb2f63e05da2e5/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:47.624214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1292/2576] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:47.624740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.626375 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.636223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:47.690383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877287a-21ef-466d-9f1e-7bf822374ce5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.693105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7df0d-022e-4faa-8cc2-3dc30c1510af admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.695000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.695713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.695998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.696276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.697012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.697959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.698620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.698743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.698928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.699202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.700210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.701948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.709033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1293/2577] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:47.712531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1285/2578] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:47.713013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.713727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.723992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38a62-c0b3-4572-9556-0a9001060c01 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.725631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90924-a321-45de-9eb0-b6cd115ec2ee admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.728745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.729631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.729914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.730192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] PATH_INFO: `/v3/projects/731a22532b9f4f84ba448705cec8f96c/users/8a1ed29cc0e044288bdb2f63e05da2e5/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.730952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614928-2181-4eaa-b314-8699031780f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.731726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614928-2181-4eaa-b314-8699031780f7 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.732012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614928-2181-4eaa-b314-8699031780f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.732628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614928-2181-4eaa-b314-8699031780f7 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.732752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731a22532b9f4f84ba448705cec8f96c, user_id=8a1ed29cc0e044288bdb2f63e05da2e5, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.733325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f614928-2181-4eaa-b314-8699031780f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.735263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f614928-2181-4eaa-b314-8699031780f7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.735406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.746653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1286/2579] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:47 2022] PUT /identity/v3/projects/731a22532b9f4f84ba448705cec8f96c/users/8a1ed29cc0e044288bdb2f63e05da2e5/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:47.747569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.768355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1294/2580] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/users => generated 287 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:47.769930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a98251-033d-4226-8602-4b3f6607c936 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.770226 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a98251-033d-4226-8602-4b3f6607c936 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.770508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a98251-033d-4226-8602-4b3f6607c936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.804982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-83a98251-033d-4226-8602-4b3f6607c936 None None] MFA Rules not processed for user `8a1ed29cc0e044288bdb2f63e05da2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:47.806219 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-83a98251-033d-4226-8602-4b3f6607c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.819222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:47.820472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab61e52-7684-4e15-bf68-597e271a3930 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.826514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.827465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.827747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.828058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] PATH_INFO: `/v3/projects/2b0fdc5cfc844da39fb82e6551e4231f/users/24b4d4fc68de44759587499eed3c8b55/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.844824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b0fdc5cfc844da39fb82e6551e4231f, user_id=24b4d4fc68de44759587499eed3c8b55, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.847605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.864354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1287/2581] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:47 2022] PUT /identity/v3/projects/2b0fdc5cfc844da39fb82e6551e4231f/users/24b4d4fc68de44759587499eed3c8b55/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:47.865578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.887024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1295/2582] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:47.888922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a98251-033d-4226-8602-4b3f6607c936 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.901917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a98251-033d-4226-8602-4b3f6607c936 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.929623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef391-8412-4659-8dab-e5455779bc88 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.934551 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.935235 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.935550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.935830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.936575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.938213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.948567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1288/2583] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:47.949322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.951180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:47.961692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.962485 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.962769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.963044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.963848 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.964650 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:47.965771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:47.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1296/2584] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:47.979985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:47.988129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a745e-143e-48e8-afba-d9e19737cbb8 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:47.993023 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:47.993689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:47.993981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:47.994291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:47.995020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:47.996745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:48.011166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1297/2585] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:48.011166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:48.015864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060683-74a1-4255-93ce-37e466fd3f1d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:48.018712 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc4c1-e461-4de2-b952-af8ca87a30b6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:48.019759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872ef53-014b-43be-a3b6-9bfc51defe6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:48.020508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872ef53-014b-43be-a3b6-9bfc51defe6c service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.020861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872ef53-014b-43be-a3b6-9bfc51defe6c service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.021202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872ef53-014b-43be-a3b6-9bfc51defe6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.023768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:48.024722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.025002 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.025376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] PATH_INFO: `/v3/projects/2b0fdc5cfc844da39fb82e6551e4231f/users/24b4d4fc68de44759587499eed3c8b55/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.027861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b0fdc5cfc844da39fb82e6551e4231f, user_id=24b4d4fc68de44759587499eed3c8b55, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:48.030967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:48.042966 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1298/2586] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:48 2022] PUT /identity/v3/projects/2b0fdc5cfc844da39fb82e6551e4231f/users/24b4d4fc68de44759587499eed3c8b55/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:48.043862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:48.083507 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872ef53-014b-43be-a3b6-9bfc51defe6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:48.085103 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872ef53-014b-43be-a3b6-9bfc51defe6c service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:48.095896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1289/2587] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:48.104532 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58463e6f-a2f5-44cb-b981-4f864d5d1492 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.104824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58463e6f-a2f5-44cb-b981-4f864d5d1492 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.105098 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58463e6f-a2f5-44cb-b981-4f864d5d1492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.108562 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18b256-9bc3-467b-a2d0-ccc3353c6691 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:48.115690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99aaeb-bac4-4426-bbc0-be7d8acc2690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:48.116075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99aaeb-bac4-4426-bbc0-be7d8acc2690 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.116355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99aaeb-bac4-4426-bbc0-be7d8acc2690 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.116629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99aaeb-bac4-4426-bbc0-be7d8acc2690 admin admin] PATH_INFO: `/v3/projects/8898548a2b15470498e3e58e3aded754/users/34d25b4b0dbc4cc59481e9a13b11bf71/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.141212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99aaeb-bac4-4426-bbc0-be7d8acc2690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8898548a2b15470498e3e58e3aded754, user_id=34d25b4b0dbc4cc59481e9a13b11bf71, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:48.144266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99aaeb-bac4-4426-bbc0-be7d8acc2690 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:48.149152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-58463e6f-a2f5-44cb-b981-4f864d5d1492 None None] MFA Rules not processed for user `24b4d4fc68de44759587499eed3c8b55`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:48.150523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-58463e6f-a2f5-44cb-b981-4f864d5d1492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:48.162840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1299/2588] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:48 2022] PUT /identity/v3/projects/8898548a2b15470498e3e58e3aded754/users/34d25b4b0dbc4cc59481e9a13b11bf71/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:48.175692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.175939 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.176201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.191175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:48.209624 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] MFA Rules not processed for user `34d25b4b0dbc4cc59481e9a13b11bf71`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:48.211855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:48.244779 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1290/2589] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:48.257598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58463e6f-a2f5-44cb-b981-4f864d5d1492 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:48.294954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1300/2590] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:48.307992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:48.309522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:48.320681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:48.330155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58463e6f-a2f5-44cb-b981-4f864d5d1492 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:48.387296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9597ae5-9038-40e8-984d-36ee53b325e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:48.388500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9597ae5-9038-40e8-984d-36ee53b325e4 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.388831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9597ae5-9038-40e8-984d-36ee53b325e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.389191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9597ae5-9038-40e8-984d-36ee53b325e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.399353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84061-a2e0-479a-9e1b-6aae20c770b8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:48.405549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:48.406121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.406404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.406721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.417548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:48.419707 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:48.430470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1301/2591] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:48.468998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9597ae5-9038-40e8-984d-36ee53b325e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:48.473575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9597ae5-9038-40e8-984d-36ee53b325e4 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:48.482556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1291/2592] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:28:48.594760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:48.687518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4d7a-9a9f-45c1-9d3b-10470cdf48d9 service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:48.692328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:48.693138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.693420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.693696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] PATH_INFO: `/v3/users/5a67c3a46833430abf18d3d890db631b` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.694930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a67c3a46833430abf18d3d890db631b)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:48.696931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:48.725875 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:48.755852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a67c3a46833430abf18d3d890db631b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:48.775173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a67c3a46833430abf18d3d890db631b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:48.785438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a67c3a46833430abf18d3d890db631b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:48.795883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a67c3a46833430abf18d3d890db631b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:48.796792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a67c3a46833430abf18d3d890db631b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:48.813347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1302/2593] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:48 2022] DELETE /identity/v3/users/5a67c3a46833430abf18d3d890db631b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:48.815089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:48.815426 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:48.815757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:48.846859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:48.848113 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:48.923357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1292/2594] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:48.933381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:48.935537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.000761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:49.001311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f0bdf-2e59-420f-a873-5ab2264f61b4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.002590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.004904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7460f06-19b8-4e01-b93c-bd6383d3a404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.005572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7460f06-19b8-4e01-b93c-bd6383d3a404 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.005855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7460f06-19b8-4e01-b93c-bd6383d3a404 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.006132 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7460f06-19b8-4e01-b93c-bd6383d3a404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.014645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ae132-16ba-4dc0-9ce8-e85ae4d26cd9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.016206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7460f06-19b8-4e01-b93c-bd6383d3a404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.017687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7460f06-19b8-4e01-b93c-bd6383d3a404 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.019318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.020233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.020521 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.020801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] PATH_INFO: `/v3/users/fb1d6fd3a7d64310bef274cfee0c7294` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.022007 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1d6fd3a7d64310bef274cfee0c7294)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.023990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.027969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1303/2595] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:49.077151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1d6fd3a7d64310bef274cfee0c7294'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.078471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:49.093589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1d6fd3a7d64310bef274cfee0c7294'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.100895 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1d6fd3a7d64310bef274cfee0c7294'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.108417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1d6fd3a7d64310bef274cfee0c7294'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.109596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1d6fd3a7d64310bef274cfee0c7294'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.120977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1293/2596] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:48 2022] DELETE /identity/v3/users/fb1d6fd3a7d64310bef274cfee0c7294 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:49.125842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.126354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.126461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.144747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:49.146071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.209416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1304/2597] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:49.220350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:49.221846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.281138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2bc2-1d53-450e-bc4e-dc2802ed9c82 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.285258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.285762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.286291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.286291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.297367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.298822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.308696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1294/2598] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:49.313998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:49.315926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.331336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c591ef-d404-4539-919c-8800e1afe31e None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.337183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.338017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.338327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.338636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] PATH_INFO: `/v3/projects/d13b1026d1dd4ef999a4086a53caccf8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.339812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d13b1026d1dd4ef999a4086a53caccf8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.341842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.387471 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:49.396758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd13b1026d1dd4ef999a4086a53caccf8', 'request_id': 'req-6c65ea9c-f463-46a7-9d54-a460c01f0569'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.407004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd13b1026d1dd4ef999a4086a53caccf8', 'request_id': 'req-6c65ea9c-f463-46a7-9d54-a460c01f0569'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.413552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd13b1026d1dd4ef999a4086a53caccf8', 'request_id': 'req-6c65ea9c-f463-46a7-9d54-a460c01f0569'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.427253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] The token cache is being invalidate because project d13b1026d1dd4ef999a4086a53caccf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:49.427577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.428383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.430220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1305/2599] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:49 2022] DELETE /identity/v3/projects/d13b1026d1dd4ef999a4086a53caccf8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:49.434153 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:49.444274 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774905c-004b-4a43-b377-348320518d56 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.507123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.508008 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.508291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.508563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] PATH_INFO: `/v3/users/8a1ed29cc0e044288bdb2f63e05da2e5` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.509792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1ed29cc0e044288bdb2f63e05da2e5)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.511776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.560659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1ed29cc0e044288bdb2f63e05da2e5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.573012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1ed29cc0e044288bdb2f63e05da2e5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.578286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:49.582811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1ed29cc0e044288bdb2f63e05da2e5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.593316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1ed29cc0e044288bdb2f63e05da2e5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.594155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1ed29cc0e044288bdb2f63e05da2e5'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.601281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1295/2600] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:49 2022] DELETE /identity/v3/users/8a1ed29cc0e044288bdb2f63e05da2e5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:49.605422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:49.606981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.624335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65ea9c-f463-46a7-9d54-a460c01f0569 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.690378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.691248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.691564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.691839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] PATH_INFO: `/v3/projects/339e6c62c08041f3a501fec502d8f22b` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.693085 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=339e6c62c08041f3a501fec502d8f22b)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.695049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.745901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '339e6c62c08041f3a501fec502d8f22b', 'request_id': 'req-44ab5f59-1604-4470-a535-6113d7367664'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.752871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '339e6c62c08041f3a501fec502d8f22b', 'request_id': 'req-44ab5f59-1604-4470-a535-6113d7367664'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.759277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '339e6c62c08041f3a501fec502d8f22b', 'request_id': 'req-44ab5f59-1604-4470-a535-6113d7367664'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.775187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] The token cache is being invalidate because project 339e6c62c08041f3a501fec502d8f22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:49.775503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.776166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.778005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1306/2601] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:49 2022] DELETE /identity/v3/projects/339e6c62c08041f3a501fec502d8f22b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:49.783405 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:49.785206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.799034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763acbb-8ca8-4cc1-8851-a06ad4cab1ba admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.862763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.863708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.863986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.864255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] PATH_INFO: `/v3/users/24b4d4fc68de44759587499eed3c8b55` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.865473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b4d4fc68de44759587499eed3c8b55)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.867448 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.910337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:49.912147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:49.930630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab5f59-1604-4470-a535-6113d7367664 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:49.938798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:49.940214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:49.940612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:49.940982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] PATH_INFO: `/v3/projects/731a22532b9f4f84ba448705cec8f96c` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:49.941640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b4d4fc68de44759587499eed3c8b55'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.943136 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=731a22532b9f4f84ba448705cec8f96c)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:49.945521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:49.958415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b4d4fc68de44759587499eed3c8b55'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.966870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b4d4fc68de44759587499eed3c8b55'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.977530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b4d4fc68de44759587499eed3c8b55'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.978219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5b980163-a941-4318-b04a-670146b0f48e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b4d4fc68de44759587499eed3c8b55'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:49.985739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1296/2602] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:49 2022] DELETE /identity/v3/users/24b4d4fc68de44759587499eed3c8b55 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:50.010391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731a22532b9f4f84ba448705cec8f96c', 'request_id': 'req-6060c3a0-56b2-4c60-9238-a1bd45040dbf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.013177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:50.019259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731a22532b9f4f84ba448705cec8f96c', 'request_id': 'req-6060c3a0-56b2-4c60-9238-a1bd45040dbf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.027583 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731a22532b9f4f84ba448705cec8f96c', 'request_id': 'req-6060c3a0-56b2-4c60-9238-a1bd45040dbf'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.053039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] The token cache is being invalidate because project 731a22532b9f4f84ba448705cec8f96c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:50.053350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.054075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6060c3a0-56b2-4c60-9238-a1bd45040dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.056134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1307/2603] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:49 2022] DELETE /identity/v3/projects/731a22532b9f4f84ba448705cec8f96c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:50.062395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bdede-0105-40a2-bdcb-2c9a1dc98539 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.062911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bdede-0105-40a2-bdcb-2c9a1dc98539 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.063020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bdede-0105-40a2-bdcb-2c9a1dc98539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.087172 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d18bdede-0105-40a2-bdcb-2c9a1dc98539 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:50.088792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d18bdede-0105-40a2-bdcb-2c9a1dc98539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:50.203416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1297/2604] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:50.212339 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382a06-f93c-410d-a885-743c27edf8eb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.212870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382a06-f93c-410d-a885-743c27edf8eb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.212977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382a06-f93c-410d-a885-743c27edf8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.234922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0a382a06-f93c-410d-a885-743c27edf8eb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:50.236218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0a382a06-f93c-410d-a885-743c27edf8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:50.244447 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1308/2605] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:50.249559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bdede-0105-40a2-bdcb-2c9a1dc98539 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.259745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bdede-0105-40a2-bdcb-2c9a1dc98539 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.264809 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.265584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.265876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.266211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.266963 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:50.267279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.269072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.270756 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:50.283556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1298/2606] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:50.288286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382a06-f93c-410d-a885-743c27edf8eb None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.299254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382a06-f93c-410d-a885-743c27edf8eb None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.305229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.306087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.306463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.306805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.307738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.309926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.320458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.322336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:50.337360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158589d-409e-4a1c-a86e-9312b52f89ca admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.343048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.344145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.344520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.344806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] PATH_INFO: `/v3/projects/2b0fdc5cfc844da39fb82e6551e4231f` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.346265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b0fdc5cfc844da39fb82e6551e4231f)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.348417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.350213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1309/2607] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:50.351199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.405390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b0fdc5cfc844da39fb82e6551e4231f', 'request_id': 'req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.412961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b0fdc5cfc844da39fb82e6551e4231f', 'request_id': 'req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.418676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b0fdc5cfc844da39fb82e6551e4231f', 'request_id': 'req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.425774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e4c65-06de-4c59-be06-26bcaae1f06b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.432945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] The token cache is being invalidate because project 2b0fdc5cfc844da39fb82e6551e4231f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:50.433317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.433967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-87d8b8a4-cd14-43e1-b88f-d1b2e75ee211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:50.435807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1299/2608] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:50 2022] DELETE /identity/v3/projects/2b0fdc5cfc844da39fb82e6551e4231f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:50.440669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd7b4-1398-4e28-8a6f-bc3425a938d5 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.441029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd7b4-1398-4e28-8a6f-bc3425a938d5 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.441316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd7b4-1398-4e28-8a6f-bc3425a938d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.459889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9e8fd7b4-1398-4e28-8a6f-bc3425a938d5 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:50.461118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9e8fd7b4-1398-4e28-8a6f-bc3425a938d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:50.489637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.490328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.490643 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.490923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.491659 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.493598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.515168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1310/2609] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:50.519697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.521513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:50.526464 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1300/2610] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:50.533867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c1c5-0832-481f-92c7-025c1561ad1a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.534948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9f29-82f5-4757-a5ef-6e57740584a6 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.535237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9f29-82f5-4757-a5ef-6e57740584a6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.535553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9f29-82f5-4757-a5ef-6e57740584a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.538661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.539340 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.539652 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.539928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.541318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.544362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.554557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c13e9f29-82f5-4757-a5ef-6e57740584a6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:50.555829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c13e9f29-82f5-4757-a5ef-6e57740584a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:50.560281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1311/2611] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:50.565444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1301/2612] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:50.568884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.569606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e9f29-82f5-4757-a5ef-6e57740584a6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.580031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e9f29-82f5-4757-a5ef-6e57740584a6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.580892 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b46bbc-04ba-4cfb-ad21-e9bd19604a47 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.585618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.585824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.585932 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.586043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.586198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.586446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.586731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.587005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:50.587130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.587311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.587770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.588972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.589363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.600633 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1312/2613] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:50.604760 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.607452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1302/2614] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 13:28:50.608622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.615166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238955ad-60e6-4c1d-9932-6e9ab483c67c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.618299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171dcf-1612-4c65-995a-f6d13dbc56ef admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.619851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.620746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.621030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.621308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] PATH_INFO: `/v3/projects/7608fbe82e1f46ef99b9837e1cee7145/users/f2de4c95534e47f192cfeeb863dde34c/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.623926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786ed93-ab7c-4df0-a830-e8224481542a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.624580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ed93-ab7c-4df0-a830-e8224481542a admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.624896 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ed93-ab7c-4df0-a830-e8224481542a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.625169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ed93-ab7c-4df0-a830-e8224481542a admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.625862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786ed93-ab7c-4df0-a830-e8224481542a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.627813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786ed93-ab7c-4df0-a830-e8224481542a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.636451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7608fbe82e1f46ef99b9837e1cee7145, user_id=f2de4c95534e47f192cfeeb863dde34c, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.639173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.658991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1313/2615] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:50 2022] PUT /identity/v3/projects/7608fbe82e1f46ef99b9837e1cee7145/users/f2de4c95534e47f192cfeeb863dde34c/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:50.659257 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1303/2616] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:50.663266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.665981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6226e6-b0ae-4f1e-8de4-4a13c830a03c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.666434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6226e6-b0ae-4f1e-8de4-4a13c830a03c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.666812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6226e6-b0ae-4f1e-8de4-4a13c830a03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.668268 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:50.702023 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9b6226e6-b0ae-4f1e-8de4-4a13c830a03c None None] MFA Rules not processed for user `f2de4c95534e47f192cfeeb863dde34c`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:50.703300 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9b6226e6-b0ae-4f1e-8de4-4a13c830a03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:50.739789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:50.743975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba05b01-c55c-41f5-913c-ea736c04863a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.805273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1304/2617] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:50.809382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6226e6-b0ae-4f1e-8de4-4a13c830a03c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.823040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.823864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.824223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.824492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.825284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.827542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.859426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1314/2618] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:28:50.860876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.871348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2935169-1fe2-4ba6-9909-d32733502d5c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.876888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.877568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.877850 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.878123 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.879018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.880653 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.892084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1315/2619] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:50.895965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6226e6-b0ae-4f1e-8de4-4a13c830a03c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.896401 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.900816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.901646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.902005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.902370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.903196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.905237 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.906501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f91ee-59fe-4e0c-b276-90afacc4a062 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:50.911192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:50.911902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:50.912187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:50.912462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:50.913198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:50.914794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:50.929157 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1316/2620] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:50.933672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.940318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1305/2621] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:50.941679 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:50.945971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfd3e6-ca8e-47ee-aeb3-08c317813e70 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.004775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d890bf9-2579-44ad-897c-5b604ef3df61 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.009639 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.009825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec513daa-2530-4520-b2a8-2a9b084a5275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.010333 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.010622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.010739 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513daa-2530-4520-b2a8-2a9b084a5275 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.010912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513daa-2530-4520-b2a8-2a9b084a5275 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.011013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.011206 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513daa-2530-4520-b2a8-2a9b084a5275 admin admin] PATH_INFO: `/v3/projects/329f8038295b45f68205e8f15f41f8d6/users/607a8cca692644e3bb5400a126ce4758/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.011768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.013787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.028897 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec513daa-2530-4520-b2a8-2a9b084a5275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329f8038295b45f68205e8f15f41f8d6, user_id=607a8cca692644e3bb5400a126ce4758, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.032381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec513daa-2530-4520-b2a8-2a9b084a5275 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.036046 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1306/2622] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:50 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:51.041195 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.046405 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:51.053419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5008-48a0-493b-b9e1-865ba58390f2 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.053883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1317/2623] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:50 2022] PUT /identity/v3/projects/329f8038295b45f68205e8f15f41f8d6/users/607a8cca692644e3bb5400a126ce4758/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:51.066111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13fbf-559f-455c-a27f-e0ba5deb4219 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.066448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13fbf-559f-455c-a27f-e0ba5deb4219 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.066826 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13fbf-559f-455c-a27f-e0ba5deb4219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.095768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-84a13fbf-559f-455c-a27f-e0ba5deb4219 None None] MFA Rules not processed for user `607a8cca692644e3bb5400a126ce4758`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:51.097029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-84a13fbf-559f-455c-a27f-e0ba5deb4219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.100204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:51.125383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.126229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.126578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.126918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.127808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.129419 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.143401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1307/2624] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:51.144567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.154496 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408ceeb-f8d5-480d-a6fd-288a3f69879a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.159109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.160039 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.160316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.160605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] PATH_INFO: `/v3/projects/c679ecb206744cb0b13a182a15797760/users/085e6d213287421db981b14002387206/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.173805 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1318/2625] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:51.178004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a13fbf-559f-455c-a27f-e0ba5deb4219 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.178162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c679ecb206744cb0b13a182a15797760, user_id=085e6d213287421db981b14002387206, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.180755 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.189554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a13fbf-559f-455c-a27f-e0ba5deb4219 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.195002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1308/2626] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:51 2022] PUT /identity/v3/projects/c679ecb206744cb0b13a182a15797760/users/085e6d213287421db981b14002387206/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:51.199069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.259504 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.260025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.260318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.260584 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.261293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.263551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.275140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf749da7-1d45-482b-a62a-3481953dc279 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.280165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.280841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.281128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.281668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.282409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.284049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.295224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1309/2627] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:51.296136 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.297960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.298672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1319/2628] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:51 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:28:51.299588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.311088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dec1b8-b11a-42c6-90d6-2d5a52fe5078 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.361605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814958c-d8c8-4eb6-8e39-c3d754244027 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.366259 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.366938 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.367221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.367522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.368220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.368827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.369716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.369997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.370160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.370290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] PATH_INFO: `/v3/users/34d25b4b0dbc4cc59481e9a13b11bf71` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.371528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d25b4b0dbc4cc59481e9a13b11bf71)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.373777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.391639 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1320/2629] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:51 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:51.392595 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.399031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:51.403866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e2e0d-b8c2-404d-beb7-e8e75ff9fab9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.411214 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.412153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.412442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.412724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] PATH_INFO: `/v3/projects/c679ecb206744cb0b13a182a15797760/users/085e6d213287421db981b14002387206/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.414751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c679ecb206744cb0b13a182a15797760, user_id=085e6d213287421db981b14002387206, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.417517 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.421465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d25b4b0dbc4cc59481e9a13b11bf71'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:51.423287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:51.429819 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1321/2630] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:51 2022] PUT /identity/v3/projects/c679ecb206744cb0b13a182a15797760/users/085e6d213287421db981b14002387206/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:51.432327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.437088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d25b4b0dbc4cc59481e9a13b11bf71'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:51.445126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d25b4b0dbc4cc59481e9a13b11bf71'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:51.455264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d25b4b0dbc4cc59481e9a13b11bf71'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:51.456063 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-bf58c4e1-d52f-4a76-9acb-a2eedc9b8848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d25b4b0dbc4cc59481e9a13b11bf71'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:51.462836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1310/2631] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:51 2022] DELETE /identity/v3/users/34d25b4b0dbc4cc59481e9a13b11bf71 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:51.464245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbaaab-9923-4ed4-a576-bff844dfd67f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.464535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbaaab-9923-4ed4-a576-bff844dfd67f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.464818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbaaab-9923-4ed4-a576-bff844dfd67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.493225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-5ecbaaab-9923-4ed4-a576-bff844dfd67f None None] MFA Rules not processed for user `085e6d213287421db981b14002387206`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:51.494549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5ecbaaab-9923-4ed4-a576-bff844dfd67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.503176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeee2ec-b71d-4212-81aa-fa5863c85098 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.558467 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.559168 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.559519 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.559833 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.560550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.562224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.564123 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1311/2632] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:51.570009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.570685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.570783 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.572790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1322/2633] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:51.576445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.578374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.592066 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:51.593309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.601698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1312/2634] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:51.602476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.603803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.616331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28db0d8-31e5-487b-892d-84bf4cc95138 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.621072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.622012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.622301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.622574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] PATH_INFO: `/v3/projects/7195f1eaa0d540adacc7c783b9eb0f56/users/87b4f58d945c4dd49c3504ca95a4a74e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.636704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2701e88-eeee-46fe-87be-fa5a68bc7a07 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.636982 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7195f1eaa0d540adacc7c783b9eb0f56, user_id=87b4f58d945c4dd49c3504ca95a4a74e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.639818 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.640577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.641233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.641513 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.641783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.653390 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.654887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.656721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1313/2635] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:51 2022] PUT /identity/v3/projects/7195f1eaa0d540adacc7c783b9eb0f56/users/87b4f58d945c4dd49c3504ca95a4a74e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:51.657543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.659229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.666084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1323/2636] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:51.667726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.678906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0c379-bec7-4cc6-ae56-41f1abef7bd7 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.716798 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:51.732995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7c889-61f9-4c48-b781-54f081887bb6 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.736782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.737489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.737774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.738105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.742726 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.743470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.743779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.744121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.744854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.746518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.747759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.749212 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.760499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1324/2637] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:51.760499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.760812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1314/2638] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:51.760919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.762634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.772365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232df36c-bd25-4832-9b9f-07f15aebca64 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.777909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27320d5-2ab0-4d24-83fc-dec1fc59f386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.778890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27320d5-2ab0-4d24-83fc-dec1fc59f386 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.779151 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27320d5-2ab0-4d24-83fc-dec1fc59f386 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.779447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27320d5-2ab0-4d24-83fc-dec1fc59f386 admin admin] PATH_INFO: `/v3/projects/7195f1eaa0d540adacc7c783b9eb0f56/users/87b4f58d945c4dd49c3504ca95a4a74e/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.781511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27320d5-2ab0-4d24-83fc-dec1fc59f386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7195f1eaa0d540adacc7c783b9eb0f56, user_id=87b4f58d945c4dd49c3504ca95a4a74e, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.784296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27320d5-2ab0-4d24-83fc-dec1fc59f386 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.799171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1315/2639] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:51 2022] PUT /identity/v3/projects/7195f1eaa0d540adacc7c783b9eb0f56/users/87b4f58d945c4dd49c3504ca95a4a74e/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:51.808945 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ced6-fd1e-4598-860e-46c274b1e518 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.809267 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ced6-fd1e-4598-860e-46c274b1e518 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.809537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ced6-fd1e-4598-860e-46c274b1e518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.835523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:51.840452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe980e9-c11e-4b6e-b9d9-0f11be9c7013 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.847799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3ef4ced6-fd1e-4598-860e-46c274b1e518 None None] MFA Rules not processed for user `87b4f58d945c4dd49c3504ca95a4a74e`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:51.849845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3ef4ced6-fd1e-4598-860e-46c274b1e518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.925765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.926549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.926917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.927388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.929420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:51.938750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1316/2640] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:51.951723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4ced6-fd1e-4598-860e-46c274b1e518 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:51.963838 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4ced6-fd1e-4598-860e-46c274b1e518 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:51.970234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:51.971277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:51.971809 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:51.972277 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:51.985303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:51.987401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:51.999886 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1317/2641] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:52.019563 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.021218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.032337 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.035490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1325/2642] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:52.085222 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.087177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:52.102522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138345c3-84e3-4bb9-97b5-cd21700d1ab6 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.108320 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.109347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.109726 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.110220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] PATH_INFO: `/v3/users/607a8cca692644e3bb5400a126ce4758` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.112465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607a8cca692644e3bb5400a126ce4758)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.116258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.162257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:52.184846 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607a8cca692644e3bb5400a126ce4758'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.185559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8242ff-4136-4634-b6ba-46276d32f7eb service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.189991 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.190930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.191205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.191541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] PATH_INFO: `/v3/projects/8898548a2b15470498e3e58e3aded754` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.192906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8898548a2b15470498e3e58e3aded754)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.194847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.202987 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607a8cca692644e3bb5400a126ce4758'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.223138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607a8cca692644e3bb5400a126ce4758'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.233584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607a8cca692644e3bb5400a126ce4758'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.234411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-1057494a-6021-4ca3-ba72-6385eca00411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607a8cca692644e3bb5400a126ce4758'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.246128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1318/2643] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:52 2022] DELETE /identity/v3/users/607a8cca692644e3bb5400a126ce4758 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:52.251009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.251627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.251627 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.259038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8898548a2b15470498e3e58e3aded754', 'request_id': 'req-fadb7652-f17e-4014-a55e-9d7ac8766597'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.265553 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:52.268082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8898548a2b15470498e3e58e3aded754', 'request_id': 'req-fadb7652-f17e-4014-a55e-9d7ac8766597'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.275888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8898548a2b15470498e3e58e3aded754', 'request_id': 'req-fadb7652-f17e-4014-a55e-9d7ac8766597'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.280580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:52.282875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:52.293165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] The token cache is being invalidate because project 8898548a2b15470498e3e58e3aded754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:52.293962 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.295398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-fadb7652-f17e-4014-a55e-9d7ac8766597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.297681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1326/2644] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:52 2022] DELETE /identity/v3/projects/8898548a2b15470498e3e58e3aded754 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:52.302696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf61ca4-f69b-4297-8387-4d77bda6fe03 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.302981 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf61ca4-f69b-4297-8387-4d77bda6fe03 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.303256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf61ca4-f69b-4297-8387-4d77bda6fe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.325239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-5bf61ca4-f69b-4297-8387-4d77bda6fe03 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:52.327227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5bf61ca4-f69b-4297-8387-4d77bda6fe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:52.379637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1319/2645] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:52.389191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.390897 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:52.431807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1327/2646] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:52.434488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762a0f4-9a18-46cc-a870-9841e58fcd4d None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.435614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762a0f4-9a18-46cc-a870-9841e58fcd4d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.436019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762a0f4-9a18-46cc-a870-9841e58fcd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.461860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7762a0f4-9a18-46cc-a870-9841e58fcd4d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:52.463573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7762a0f4-9a18-46cc-a870-9841e58fcd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:52.472033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbef06-7fe6-4315-bdc4-3b6a197acecc None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.476733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1328/2647] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:52.477956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762a0f4-9a18-46cc-a870-9841e58fcd4d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.479608 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.480307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.480581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.480862 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.489691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762a0f4-9a18-46cc-a870-9841e58fcd4d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.492224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.493775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.495613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.496380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.496811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.497176 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.498738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:52.499105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.500871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.505881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1320/2648] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:28:52.515530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1329/2649] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:52.520245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.534010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4fa0e-ac95-46e9-b2dd-b2c82cad7517 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.539177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.539956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.540255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.540530 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.541229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.543254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.578326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1321/2650] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:52 2022] POST /identity/v3/projects => generated 357 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:52.583272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.595049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616c3e5-4f9f-4155-ad2c-95471fa9225e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.650568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:52.659240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.660051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.660439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.661194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.661946 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.663997 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.690229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1330/2651] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:52 2022] POST /identity/v3/users => generated 279 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:28:52.694391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.706510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197ea04-b154-4032-a968-e90e2a5d0b59 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.711778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.712584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.712938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.713302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.714826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.716973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.728201 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1322/2652] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:52.733111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.743538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da8d10-0089-4348-a1d9-c8ae6d2e4094 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.748995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.749811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.750098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.750372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.751108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.752728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.763694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1331/2653] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:52.768904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.779885 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae005f-c753-45ed-9b2c-2b36c9952d33 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.784622 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c60bf-b8b5-4470-8a7e-90ea7c94db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.785575 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c60bf-b8b5-4470-8a7e-90ea7c94db22 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.785827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c60bf-b8b5-4470-8a7e-90ea7c94db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.786101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c60bf-b8b5-4470-8a7e-90ea7c94db22 admin admin] PATH_INFO: `/v3/projects/cb08466c0bed478785a2330ac19477e1/users/c5b7e36490ce4b889de3edda9d22d9f8/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.802312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c60bf-b8b5-4470-8a7e-90ea7c94db22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb08466c0bed478785a2330ac19477e1, user_id=c5b7e36490ce4b889de3edda9d22d9f8, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.805009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c60bf-b8b5-4470-8a7e-90ea7c94db22 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.810108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.812940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:52.824498 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1323/2654] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:52 2022] PUT /identity/v3/projects/cb08466c0bed478785a2330ac19477e1/users/c5b7e36490ce4b889de3edda9d22d9f8/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:52.827541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6371886-1089-40d3-9a1c-9c989321a556 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.839477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463647-7b0a-40fa-8835-9e1b5f08ab37 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.839772 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463647-7b0a-40fa-8835-9e1b5f08ab37 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.840087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463647-7b0a-40fa-8835-9e1b5f08ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.869005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9f463647-7b0a-40fa-8835-9e1b5f08ab37 None None] MFA Rules not processed for user `c5b7e36490ce4b889de3edda9d22d9f8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:52.870261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9f463647-7b0a-40fa-8835-9e1b5f08ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:52.892257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:52.893141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:52.893418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:52.893687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] PATH_INFO: `/v3/projects/329f8038295b45f68205e8f15f41f8d6` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:52.895266 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329f8038295b45f68205e8f15f41f8d6)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:52.897225 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:52.909630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:52.939160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1324/2655] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:52.944036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f463647-7b0a-40fa-8835-9e1b5f08ab37 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.950961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329f8038295b45f68205e8f15f41f8d6', 'request_id': 'req-ccb26478-0c8f-4bd8-a447-b52bf50de418'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.955210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f463647-7b0a-40fa-8835-9e1b5f08ab37 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:52.957756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329f8038295b45f68205e8f15f41f8d6', 'request_id': 'req-ccb26478-0c8f-4bd8-a447-b52bf50de418'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.963856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329f8038295b45f68205e8f15f41f8d6', 'request_id': 'req-ccb26478-0c8f-4bd8-a447-b52bf50de418'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.977462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] The token cache is being invalidate because project 329f8038295b45f68205e8f15f41f8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:52.977756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.978448 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:52.980080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1332/2656] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:52 2022] DELETE /identity/v3/projects/329f8038295b45f68205e8f15f41f8d6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:52.984714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:52.986184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.011926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.012842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.013339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.013631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.014322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.016271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.044093 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1325/2657] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:52 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:28:53.047943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.049685 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.053329 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb26478-0c8f-4bd8-a447-b52bf50de418 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.061792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:53.117486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1d22e-004f-430d-94f9-4634e7b969f1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.118508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.119456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.119744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.120015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] PATH_INFO: `/v3/users/87b4f58d945c4dd49c3504ca95a4a74e` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.121231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b4f58d945c4dd49c3504ca95a4a74e)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.123283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.123757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.124000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.124292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.124565 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.125431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.127199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.150213 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1326/2658] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:53 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:53.154655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.165154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929f67-0c86-43c3-b330-69edc6a4f0f8 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.170158 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.170835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.171124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.171537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.172335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.174074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.175420 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b4f58d945c4dd49c3504ca95a4a74e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.187887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1327/2659] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:28:53.188790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b4f58d945c4dd49c3504ca95a4a74e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.189005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.197315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b4f58d945c4dd49c3504ca95a4a74e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.200196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf62bf2-23aa-4be4-9023-3604d01e6e1b admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.204999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b4f58d945c4dd49c3504ca95a4a74e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.205155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.205719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b4f58d945c4dd49c3504ca95a4a74e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.205904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.206193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.206477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] PATH_INFO: `/v3/projects/387127c805ca4aa593f0c37be0c6ff58/users/5397622ec534492dadbf95a786765b3f/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.211839 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1333/2660] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:52 2022] DELETE /identity/v3/users/87b4f58d945c4dd49c3504ca95a4a74e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:53.222363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387127c805ca4aa593f0c37be0c6ff58, user_id=5397622ec534492dadbf95a786765b3f, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.225245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.239687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1328/2661] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:53 2022] PUT /identity/v3/projects/387127c805ca4aa593f0c37be0c6ff58/users/5397622ec534492dadbf95a786765b3f/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:53.244046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.245666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.316407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ab949-6950-4402-9ed5-b76b144132be admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.353786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:53.382837 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029b9c-c662-42f2-9a5f-9ef7929f72c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.383561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029b9c-c662-42f2-9a5f-9ef7929f72c9 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.383847 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029b9c-c662-42f2-9a5f-9ef7929f72c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.384128 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029b9c-c662-42f2-9a5f-9ef7929f72c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.384956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029b9c-c662-42f2-9a5f-9ef7929f72c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.386516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029b9c-c662-42f2-9a5f-9ef7929f72c9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.397188 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1334/2662] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:53.401766 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.412632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881c71-2a7e-4e4c-82a0-b64547157295 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.417918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.418778 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.419079 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.419355 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] PATH_INFO: `/v3/projects/387127c805ca4aa593f0c37be0c6ff58/users/5397622ec534492dadbf95a786765b3f/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.421445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387127c805ca4aa593f0c37be0c6ff58, user_id=5397622ec534492dadbf95a786765b3f, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.424191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.435310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1329/2663] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:53 2022] PUT /identity/v3/projects/387127c805ca4aa593f0c37be0c6ff58/users/5397622ec534492dadbf95a786765b3f/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:53.445980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.446280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.446567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.479526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] MFA Rules not processed for user `5397622ec534492dadbf95a786765b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:53.480771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.525162 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.530547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.538613 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:53.581546 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1335/2664] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:53 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:53.595878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.595878 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.628761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d9e2e-a77b-4da1-8afa-765ecac83f93 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.634708 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.635926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.636439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.636718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] PATH_INFO: `/v3/projects/7195f1eaa0d540adacc7c783b9eb0f56` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.638043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7195f1eaa0d540adacc7c783b9eb0f56)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.640074 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.674874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeae1d-9a64-4f19-9957-def6c813b97b None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.679580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c789d-b265-4a20-9016-d7435184c9ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.680683 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c789d-b265-4a20-9016-d7435184c9ca service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.681292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c789d-b265-4a20-9016-d7435184c9ca service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.681872 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c789d-b265-4a20-9016-d7435184c9ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.695472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c789d-b265-4a20-9016-d7435184c9ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.696930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c789d-b265-4a20-9016-d7435184c9ca service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.702721 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7195f1eaa0d540adacc7c783b9eb0f56', 'request_id': 'req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.707898 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1336/2665] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:53 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:53.709569 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7195f1eaa0d540adacc7c783b9eb0f56', 'request_id': 'req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.717019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7195f1eaa0d540adacc7c783b9eb0f56', 'request_id': 'req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.731793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] The token cache is being invalidate because project 7195f1eaa0d540adacc7c783b9eb0f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:53.732372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.733070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:53.735412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1330/2666] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:53 2022] DELETE /identity/v3/projects/7195f1eaa0d540adacc7c783b9eb0f56 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:53.741416 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8918244-2e92-4e1f-87e9-c40543902146 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.741920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8918244-2e92-4e1f-87e9-c40543902146 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.742212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8918244-2e92-4e1f-87e9-c40543902146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.762804 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a8918244-2e92-4e1f-87e9-c40543902146 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:53.764404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a8918244-2e92-4e1f-87e9-c40543902146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.788343 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.790266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.839745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1337/2667] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:53.848256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68787c-cd82-4061-98e5-1e33a9e505c3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.848751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68787c-cd82-4061-98e5-1e33a9e505c3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.848988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68787c-cd82-4061-98e5-1e33a9e505c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.856777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:53.869831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a09c7-182d-4217-b03e-b76c5f1f73b9 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.871636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9b68787c-cd82-4061-98e5-1e33a9e505c3 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:53.873315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9b68787c-cd82-4061-98e5-1e33a9e505c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.876555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.878388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.878761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.879044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.881664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:53.882989 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1338/2668] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:53.890097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68787c-cd82-4061-98e5-1e33a9e505c3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.900251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68787c-cd82-4061-98e5-1e33a9e505c3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.906239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.906724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.907097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.907476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.908294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:53.908676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.910357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.924924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1339/2669] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:53.929862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:53.938428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:53.948597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0d90-13b7-4cd1-8a6f-75a6d17a2fbf admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:53.965164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:53.967617 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:53.967954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:53.968281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:53.968918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.971462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:53.972198 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:53.974371 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.006893 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1331/2670] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:53 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:54.011442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1340/2671] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:53 2022] POST /identity/v3/projects => generated 401 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:54.017000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.041330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd3962-d937-47c6-ad8a-cda2fced5d09 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.103592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.104434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.105388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.105388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.105921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.108678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.139833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1332/2672] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:54 2022] POST /identity/v3/users => generated 301 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:54.141887 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.154297 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd74c-b2f1-4068-a0f2-867880b9bda5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.159443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.160330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.160618 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.161018 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.161967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.163674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.177930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1341/2673] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:54.183678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.195554 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f768a-1f50-43f2-ab28-0329cb5138a9 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.200542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b95dbf-66ac-4582-8e63-fac017eb32d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.201258 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b95dbf-66ac-4582-8e63-fac017eb32d6 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.201545 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b95dbf-66ac-4582-8e63-fac017eb32d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.201826 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b95dbf-66ac-4582-8e63-fac017eb32d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.202572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b95dbf-66ac-4582-8e63-fac017eb32d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.204288 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b95dbf-66ac-4582-8e63-fac017eb32d6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.215043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1333/2674] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:54.219697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.230489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e705aa6-e4f8-496e-9934-2e06cdaae63c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.235666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.236569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.236856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.237133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] PATH_INFO: `/v3/projects/b73e128d19c54270834799c1412f034d/users/21d12d09a2fc4c179c355e32c4d03890/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.253069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73e128d19c54270834799c1412f034d, user_id=21d12d09a2fc4c179c355e32c4d03890, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.255771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.268906 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1342/2675] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:54 2022] PUT /identity/v3/projects/b73e128d19c54270834799c1412f034d/users/21d12d09a2fc4c179c355e32c4d03890/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:54.280919 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264331e-8485-4fac-b7ce-cbdffd18d210 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.281437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264331e-8485-4fac-b7ce-cbdffd18d210 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.281735 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264331e-8485-4fac-b7ce-cbdffd18d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.311557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c264331e-8485-4fac-b7ce-cbdffd18d210 None None] MFA Rules not processed for user `21d12d09a2fc4c179c355e32c4d03890`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:54.312781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c264331e-8485-4fac-b7ce-cbdffd18d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:54.387523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1334/2676] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:28:54.392501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.458017 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd9da1-8077-4209-a57e-bcfd35ecfe80 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.463028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.463721 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.464009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.464292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.464992 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.467794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.497891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1343/2677] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:54 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:28:54.499631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264331e-8485-4fac-b7ce-cbdffd18d210 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.510482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264331e-8485-4fac-b7ce-cbdffd18d210 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.563417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:54.570748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.571495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.571757 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.572029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.572774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.574740 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.596681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1335/2678] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:54 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:54.600853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.611954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77cc0b-50b1-472b-9e32-410e91a30db3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.616877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.617540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.617844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.618119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.618851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.620473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.631589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1344/2679] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:54.635817 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.646992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810d54a-0e7d-48de-b246-25165100fd7c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.651928 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.652847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.653185 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.653486 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] PATH_INFO: `/v3/projects/b5a972256fa64dc9b246fcdab0ec5676/users/779749b6b9e5484ea657aaca8182287c/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.668737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a972256fa64dc9b246fcdab0ec5676, user_id=779749b6b9e5484ea657aaca8182287c, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.671415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.686176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1336/2680] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:54 2022] PUT /identity/v3/projects/b5a972256fa64dc9b246fcdab0ec5676/users/779749b6b9e5484ea657aaca8182287c/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:54.690235 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.701089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d19f2-4487-4d3a-9ddd-b443f16cde7f admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.765905 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0aab2-fa7e-418f-9f86-9fa598a371a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.766593 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0aab2-fa7e-418f-9f86-9fa598a371a8 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.766877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0aab2-fa7e-418f-9f86-9fa598a371a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.767177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0aab2-fa7e-418f-9f86-9fa598a371a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.767987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0aab2-fa7e-418f-9f86-9fa598a371a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.769656 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0aab2-fa7e-418f-9f86-9fa598a371a8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.771040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:54.782124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1345/2681] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:54.786242 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:54.797200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83937590-6733-4a37-bf1a-731f72ebf46c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:54.801992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:54.802894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.803202 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.803505 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] PATH_INFO: `/v3/projects/b5a972256fa64dc9b246fcdab0ec5676/users/779749b6b9e5484ea657aaca8182287c/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.805459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a972256fa64dc9b246fcdab0ec5676, user_id=779749b6b9e5484ea657aaca8182287c, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:54.808155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:54.819361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1337/2682] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:54 2022] PUT /identity/v3/projects/b5a972256fa64dc9b246fcdab0ec5676/users/779749b6b9e5484ea657aaca8182287c/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:54.830713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cfe5-6eb4-4c52-a25a-ad424bc00003 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:54.831038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cfe5-6eb4-4c52-a25a-ad424bc00003 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:54.831319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cfe5-6eb4-4c52-a25a-ad424bc00003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:54.860502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9248cfe5-6eb4-4c52-a25a-ad424bc00003 None None] MFA Rules not processed for user `779749b6b9e5484ea657aaca8182287c`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:54.861731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9248cfe5-6eb4-4c52-a25a-ad424bc00003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:54.932919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1346/2683] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:54.944332 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:55.003085 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2d256-7767-4e70-a24e-5ee1d832120e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:55.007152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:55.007916 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:55.008203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:55.008483 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:55.018514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:55.020049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:55.030572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1338/2684] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:55.115869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248cfe5-6eb4-4c52-a25a-ad424bc00003 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:55.130469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248cfe5-6eb4-4c52-a25a-ad424bc00003 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:55.134582 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c2d26-98a8-4426-a5af-214e46140393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:55.135285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c2d26-98a8-4426-a5af-214e46140393 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:55.135605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c2d26-98a8-4426-a5af-214e46140393 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:55.135882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c2d26-98a8-4426-a5af-214e46140393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:55.202649 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c2d26-98a8-4426-a5af-214e46140393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:55.204165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c2d26-98a8-4426-a5af-214e46140393 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:55.214646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1347/2685] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:55.767391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:55.769326 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:55.886356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd757b-0424-4fc4-95b1-afebb3719c69 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:55.891229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:55.892155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:55.892441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:55.892718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] PATH_INFO: `/v3/users/f2de4c95534e47f192cfeeb863dde34c` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:55.893950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2de4c95534e47f192cfeeb863dde34c)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:55.895931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:55.925736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:55.947203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2de4c95534e47f192cfeeb863dde34c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:55.958934 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2de4c95534e47f192cfeeb863dde34c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:55.965833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2de4c95534e47f192cfeeb863dde34c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:55.974670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2de4c95534e47f192cfeeb863dde34c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:55.975351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2de4c95534e47f192cfeeb863dde34c'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:55.981805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1339/2686] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:55 2022] DELETE /identity/v3/users/f2de4c95534e47f192cfeeb863dde34c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:55.986834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:55.987361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:55.987361 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:56.004678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:56.005890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:56.068135 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1348/2687] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:56.078330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:56.079845 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:56.139913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792cb6-e228-48f0-b295-32b9ba30f16e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:56.143927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:56.144594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:56.144876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:56.145140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:56.154782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:56.156312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:56.165931 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1340/2688] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:56.443880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:56.445987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:56.461379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c309bc-f9c8-4e18-8688-1f182d3f36e6 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:56.467344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:56.468356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:56.468731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:56.468912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] PATH_INFO: `/v3/projects/7608fbe82e1f46ef99b9837e1cee7145` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:56.470220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7608fbe82e1f46ef99b9837e1cee7145)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:56.472208 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:56.490091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=100 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:56.521917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7608fbe82e1f46ef99b9837e1cee7145', 'request_id': 'req-1186b93a-d316-4b62-a5c1-d6b1b08dae53'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.528452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7608fbe82e1f46ef99b9837e1cee7145', 'request_id': 'req-1186b93a-d316-4b62-a5c1-d6b1b08dae53'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.534228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7608fbe82e1f46ef99b9837e1cee7145', 'request_id': 'req-1186b93a-d316-4b62-a5c1-d6b1b08dae53'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.549438 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] The token cache is being invalidate because project 7608fbe82e1f46ef99b9837e1cee7145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:56.549733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.550851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.552485 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1349/2689] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:56 2022] DELETE /identity/v3/projects/7608fbe82e1f46ef99b9837e1cee7145 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:56.562933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:56.567269 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b0191-f315-4ab3-a36d-dfb9476d2170 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:56.643563 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:56.644466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:56.644754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:56.645031 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] PATH_INFO: `/v3/users/085e6d213287421db981b14002387206` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:56.646259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] RBAC: Authorizing `identity:delete_user(user_id=085e6d213287421db981b14002387206)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:56.648252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:56.701317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '085e6d213287421db981b14002387206'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.703474 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:56.721417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '085e6d213287421db981b14002387206'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.729542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '085e6d213287421db981b14002387206'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.739810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '085e6d213287421db981b14002387206'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.740790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-34b77458-acde-485f-9722-3e233c2e475e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '085e6d213287421db981b14002387206'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:56.751676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1341/2690] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:56 2022] DELETE /identity/v3/users/085e6d213287421db981b14002387206 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:57.117930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.119591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:57.136054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186b93a-d316-4b62-a5c1-d6b1b08dae53 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.201759 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.202564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.202849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.203126 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] PATH_INFO: `/v3/projects/c679ecb206744cb0b13a182a15797760` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.204395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c679ecb206744cb0b13a182a15797760)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.206681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.257692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c679ecb206744cb0b13a182a15797760', 'request_id': 'req-c711c134-226c-422b-8561-f8e5dea13a02'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:57.264353 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c679ecb206744cb0b13a182a15797760', 'request_id': 'req-c711c134-226c-422b-8561-f8e5dea13a02'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:57.269976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c679ecb206744cb0b13a182a15797760', 'request_id': 'req-c711c134-226c-422b-8561-f8e5dea13a02'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:57.283526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] The token cache is being invalidate because project c679ecb206744cb0b13a182a15797760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:28:57.283825 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:57.284460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-c711c134-226c-422b-8561-f8e5dea13a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:57.286088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1350/2691] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:28:57 2022] DELETE /identity/v3/projects/c679ecb206744cb0b13a182a15797760 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:57.291220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d812135-b4a7-48ba-a0c6-0993c0ba75ad None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.291748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d812135-b4a7-48ba-a0c6-0993c0ba75ad None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.291853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d812135-b4a7-48ba-a0c6-0993c0ba75ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.311128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-5d812135-b4a7-48ba-a0c6-0993c0ba75ad None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:57.312356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5d812135-b4a7-48ba-a0c6-0993c0ba75ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:57.378182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1342/2692] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:57.387410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97f747-1609-465c-a4b3-42e2b31f60c4 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.387704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97f747-1609-465c-a4b3-42e2b31f60c4 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.388006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97f747-1609-465c-a4b3-42e2b31f60c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.407828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1f97f747-1609-465c-a4b3-42e2b31f60c4 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:57.409039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1f97f747-1609-465c-a4b3-42e2b31f60c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:57.418576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1351/2693] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:57.423102 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d812135-b4a7-48ba-a0c6-0993c0ba75ad None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.433396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d812135-b4a7-48ba-a0c6-0993c0ba75ad None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.438581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.439406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.439713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.439988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.440734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:28:57.441040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.442672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.454898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1343/2694] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:28:57.459275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97f747-1609-465c-a4b3-42e2b31f60c4 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.469544 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97f747-1609-465c-a4b3-42e2b31f60c4 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.475058 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.475767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.476052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.476325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.477030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.478949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.506688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1352/2695] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:57 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:57.510992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.522484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b959b-5f94-4495-bfcc-159ac604da16 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.583995 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.585059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.585348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.585626 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.586331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.588345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.610247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1344/2696] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:57 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:57.614577 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.621427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:57.626508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bbf15-4535-4762-bb11-ed0633801747 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.633203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.633945 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.634398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.634733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.635635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.638029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.649281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1353/2697] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:57.653559 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.664592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721f061-7578-4620-a201-32bd49fa5da9 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.670417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea3894-1cf9-4b31-864e-78badc33f16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.671100 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea3894-1cf9-4b31-864e-78badc33f16c admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.671400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea3894-1cf9-4b31-864e-78badc33f16c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.671759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea3894-1cf9-4b31-864e-78badc33f16c admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.672487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea3894-1cf9-4b31-864e-78badc33f16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.674112 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea3894-1cf9-4b31-864e-78badc33f16c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.685015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1345/2698] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:57.689432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.700228 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14f79-4422-4d1c-af44-714306168da5 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.705275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.706095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.706378 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.706673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] PATH_INFO: `/v3/projects/01a98b3c25264b86ae1e016303015650/users/aca063167cd64b71b422e24121b13434/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.722256 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a98b3c25264b86ae1e016303015650, user_id=aca063167cd64b71b422e24121b13434, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.724961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.739304 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1354/2699] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:57 2022] PUT /identity/v3/projects/01a98b3c25264b86ae1e016303015650/users/aca063167cd64b71b422e24121b13434/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:28:57.744952 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04a37b-fef8-48b7-9289-deb016513b27 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.745266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04a37b-fef8-48b7-9289-deb016513b27 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.745548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04a37b-fef8-48b7-9289-deb016513b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.766078 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8a04a37b-fef8-48b7-9289-deb016513b27 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:57.767243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8a04a37b-fef8-48b7-9289-deb016513b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:57.793723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:57.831847 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1346/2700] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:57.841816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:57.844144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:57.912270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390a8b3-42bc-47ba-94f1-ea03904fa79e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:57.916380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:57.917034 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:57.917317 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:57.917588 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:57.927197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:57.928687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:57.938933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1355/2701] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:28:58.667234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a4b3-74ff-4eda-b74e-994f4dfacdbd None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:58.668184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a4b3-74ff-4eda-b74e-994f4dfacdbd None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:58.668676 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a4b3-74ff-4eda-b74e-994f4dfacdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:58.701444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-8161a4b3-74ff-4eda-b74e-994f4dfacdbd None None] MFA Rules not processed for user `aca063167cd64b71b422e24121b13434`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:58.702160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8161a4b3-74ff-4eda-b74e-994f4dfacdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:58.771192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1347/2702] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:58.776538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:58.786952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929531aa-8745-4ad0-b4ac-f0762943c77d service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:58.791927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:58.792676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:58.793026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:58.793367 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:58.794164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:58.796216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:58.828967 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1356/2703] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:28:58 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:28:58.830736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161a4b3-74ff-4eda-b74e-994f4dfacdbd None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:58.843677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161a4b3-74ff-4eda-b74e-994f4dfacdbd None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:58.912376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:58.913082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:58.913367 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:58.913645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:58.914356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:58.916357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:58.939638 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1348/2704] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:28:58 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:28:58.944005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:58.957180 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3798e66-44a7-4f62-a8a8-4235a30f89a3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:58.966444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:58.966711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:58.966827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:58.967769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:58.968052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:58.969686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:58.984024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1357/2705] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:58.989509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:59.000892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590545-631a-47b3-b6ab-37dc3c1d646a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:59.008193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:59.009245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:59.009585 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:59.009918 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] PATH_INFO: `/v3/projects/5878e03801294ffd825603b4ae70d2e2/users/010105791cdb4110bc848141920140f8/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:59.027121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5878e03801294ffd825603b4ae70d2e2, user_id=010105791cdb4110bc848141920140f8, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:59.029821 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:59.045511 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1349/2706] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:58 2022] PUT /identity/v3/projects/5878e03801294ffd825603b4ae70d2e2/users/010105791cdb4110bc848141920140f8/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:59.050131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:59.063998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8ba53-945b-4d29-97bb-6ed3a14e501c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:59.124856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:59.125789 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:59.126255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:59.126725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:59.127689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:59.129490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:59.140867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1358/2707] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:28:59.145353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:59.156564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00c07-777b-4964-af5a-518befdb7721 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:59.164253 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e65dd-4a5b-40c3-9d90-da49a4bbc2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:59.165165 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e65dd-4a5b-40c3-9d90-da49a4bbc2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:59.165452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e65dd-4a5b-40c3-9d90-da49a4bbc2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:59.165730 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e65dd-4a5b-40c3-9d90-da49a4bbc2a3 admin admin] PATH_INFO: `/v3/projects/5878e03801294ffd825603b4ae70d2e2/users/010105791cdb4110bc848141920140f8/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:59.168057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e65dd-4a5b-40c3-9d90-da49a4bbc2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5878e03801294ffd825603b4ae70d2e2, user_id=010105791cdb4110bc848141920140f8, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:59.171993 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e65dd-4a5b-40c3-9d90-da49a4bbc2a3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:59.184372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1350/2708] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:28:59 2022] PUT /identity/v3/projects/5878e03801294ffd825603b4ae70d2e2/users/010105791cdb4110bc848141920140f8/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:59.492699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:59.494499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:59.507681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e27cf1-c989-4d87-bd80-bfbe0fd8ce90 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:59.554560 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:59.585829 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:59.587127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:59.587435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:59.587734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] PATH_INFO: `/v3/users/21d12d09a2fc4c179c355e32c4d03890` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:59.589554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d12d09a2fc4c179c355e32c4d03890)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:59.592183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:28:59.654299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d12d09a2fc4c179c355e32c4d03890'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:59.669492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d12d09a2fc4c179c355e32c4d03890'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:59.678685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d12d09a2fc4c179c355e32c4d03890'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:59.688730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d12d09a2fc4c179c355e32c4d03890'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:59.689638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d12d09a2fc4c179c355e32c4d03890'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:28:59.706895 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1359/2709] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:28:59 2022] DELETE /identity/v3/users/21d12d09a2fc4c179c355e32c4d03890 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:28:59.707786 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf93212-bf08-4fa3-94ff-6f8179fa0c18 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:59.708324 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf93212-bf08-4fa3-94ff-6f8179fa0c18 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:59.708562 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf93212-bf08-4fa3-94ff-6f8179fa0c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:59.736670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-fdf93212-bf08-4fa3-94ff-6f8179fa0c18 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:59.738322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fdf93212-bf08-4fa3-94ff-6f8179fa0c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:59.755509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:28:59.832790 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1351/2710] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:28:59.845081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:28:59.855998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:59.895853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:59.896318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:59.896787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:59.944471 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] MFA Rules not processed for user `010105791cdb4110bc848141920140f8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:28:59.946660 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:28:59.973044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5f22-2a98-4114-83af-be43d6e11413 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:28:59.977125 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:28:59.977812 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:28:59.978100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:28:59.978381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:28:59.989931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:28:59.991491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:00.008689 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1360/2711] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:00.050259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1352/2712] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:28:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:00.062152 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:00.063697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:00.164715 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:00.172175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5e889-ac43-416c-8ee2-cb72c5b14a23 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:00.178111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:00.178968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:00.180211 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:00.180576 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:00.182854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:00.256093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:00.257567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:00.268099 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1361/2713] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:00.400727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:00.400727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:00.400727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ed062-ad1e-4a55-a660-955e295aa482 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:00.401444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:00.401444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:00.401444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:00.401444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] PATH_INFO: `/v3/projects/b73e128d19c54270834799c1412f034d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:00.401892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73e128d19c54270834799c1412f034d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:00.401892 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:00.463064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73e128d19c54270834799c1412f034d', 'request_id': 'req-eb03cde1-8dec-4bbf-ae7d-6868248595d0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.471947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73e128d19c54270834799c1412f034d', 'request_id': 'req-eb03cde1-8dec-4bbf-ae7d-6868248595d0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.478256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73e128d19c54270834799c1412f034d', 'request_id': 'req-eb03cde1-8dec-4bbf-ae7d-6868248595d0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.496870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] The token cache is being invalidate because project b73e128d19c54270834799c1412f034d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:00.497159 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.497814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.499396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1353/2714] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:00 2022] DELETE /identity/v3/projects/b73e128d19c54270834799c1412f034d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:00.503909 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:00.516861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed3f24-6b74-45d0-9528-42aeff07d88e service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:00.593231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:00.594159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:00.594443 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:00.594722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] PATH_INFO: `/v3/users/779749b6b9e5484ea657aaca8182287c` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:00.596106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779749b6b9e5484ea657aaca8182287c)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:00.598166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:00.668137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779749b6b9e5484ea657aaca8182287c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.685237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779749b6b9e5484ea657aaca8182287c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.695882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779749b6b9e5484ea657aaca8182287c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.706144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779749b6b9e5484ea657aaca8182287c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.706849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-1e2c5060-8cdb-40f7-98d4-c4e6d05c05c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779749b6b9e5484ea657aaca8182287c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:00.713041 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1362/2715] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:00 2022] DELETE /identity/v3/users/779749b6b9e5484ea657aaca8182287c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:01.028531 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.030871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:01.051224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03cde1-8dec-4bbf-ae7d-6868248595d0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.072584 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:01.118600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.119655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.120036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.120409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] PATH_INFO: `/v3/projects/b5a972256fa64dc9b246fcdab0ec5676` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.122200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a972256fa64dc9b246fcdab0ec5676)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.124773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.185603 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a972256fa64dc9b246fcdab0ec5676', 'request_id': 'req-13805ad5-a1c1-469c-9541-268bbbba9f09'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:01.194799 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a972256fa64dc9b246fcdab0ec5676', 'request_id': 'req-13805ad5-a1c1-469c-9541-268bbbba9f09'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:01.202495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a972256fa64dc9b246fcdab0ec5676', 'request_id': 'req-13805ad5-a1c1-469c-9541-268bbbba9f09'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:01.219494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] The token cache is being invalidate because project b5a972256fa64dc9b246fcdab0ec5676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:01.219930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:01.223518 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-13805ad5-a1c1-469c-9541-268bbbba9f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:01.225126 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1354/2716] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:01 2022] DELETE /identity/v3/projects/b5a972256fa64dc9b246fcdab0ec5676 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:01.230890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36383-1329-445f-b8c1-5d7e3efe5a25 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.231558 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36383-1329-445f-b8c1-5d7e3efe5a25 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.231822 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36383-1329-445f-b8c1-5d7e3efe5a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.252075 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-2ef36383-1329-445f-b8c1-5d7e3efe5a25 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:01.257807 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2ef36383-1329-445f-b8c1-5d7e3efe5a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:01.268953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:01.325824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1363/2717] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:01.335334 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613915-ed33-4042-a31a-033a9c3b64cc None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.335658 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613915-ed33-4042-a31a-033a9c3b64cc None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.335943 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613915-ed33-4042-a31a-033a9c3b64cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.355664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-38613915-ed33-4042-a31a-033a9c3b64cc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:01.357364 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-38613915-ed33-4042-a31a-033a9c3b64cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:01.369029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1355/2718] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:01.372542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef36383-1329-445f-b8c1-5d7e3efe5a25 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.383435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef36383-1329-445f-b8c1-5d7e3efe5a25 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.388703 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.389432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.389696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.389978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.390738 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:01.391081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.392788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.406246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1364/2719] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:01.411122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38613915-ed33-4042-a31a-033a9c3b64cc None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.423024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38613915-ed33-4042-a31a-033a9c3b64cc None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.428446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.429310 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.429610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.429887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.430594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.432712 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.464453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1356/2720] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:01 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:01.469150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.479311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0fbdb-2467-42db-9410-b51deacfd797 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.542674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.543387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.543773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.544061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.544809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.546728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.570073 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1365/2721] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:01 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:01.574986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.589630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ffec-50ad-4d3d-8799-a93be7e536b3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.597673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.598414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.598700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.598976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.599771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.601949 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.616997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1357/2722] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:01.621854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.633177 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d2be0-c176-4ebf-9ada-b1b640dbac59 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.639408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.640124 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.640393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.640668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.641418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.643061 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.656559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1366/2723] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:01.661271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.672460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b79a-6077-4f0c-bfc1-eaa7e4c846aa admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.678519 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7c6f1-ee62-4a83-926c-7d05283b10ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.679683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c6f1-ee62-4a83-926c-7d05283b10ca admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.680037 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c6f1-ee62-4a83-926c-7d05283b10ca admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.680387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c6f1-ee62-4a83-926c-7d05283b10ca admin admin] PATH_INFO: `/v3/projects/8e28e29f9679406aaf7038b4291e3106/users/e8a615ea755d4224a7b91858262531e5/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.697688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7c6f1-ee62-4a83-926c-7d05283b10ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e28e29f9679406aaf7038b4291e3106, user_id=e8a615ea755d4224a7b91858262531e5, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.700625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7c6f1-ee62-4a83-926c-7d05283b10ca admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.701776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.704811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:01.717359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1358/2724] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:01 2022] PUT /identity/v3/projects/8e28e29f9679406aaf7038b4291e3106/users/e8a615ea755d4224a7b91858262531e5/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:01.728017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb7917-62d1-410e-a0cd-8b4fcba9112d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.728312 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb7917-62d1-410e-a0cd-8b4fcba9112d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.728592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb7917-62d1-410e-a0cd-8b4fcba9112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.758266 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-7afb7917-62d1-410e-a0cd-8b4fcba9112d None None] MFA Rules not processed for user `e8a615ea755d4224a7b91858262531e5`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:01.759601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7afb7917-62d1-410e-a0cd-8b4fcba9112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:01.769944 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416241d-1abc-4315-9743-127ffb5da20c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.799006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:01.813253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:01.824407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.825567 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.826844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.827205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.830045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:01.835739 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1359/2725] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:01.840418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb7917-62d1-410e-a0cd-8b4fcba9112d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.897801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.899197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.906694 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb7917-62d1-410e-a0cd-8b4fcba9112d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:01.909227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1367/2726] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:01.911666 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:01.912365 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:01.912629 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:01.912915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:01.913673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:01.915635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:01.945706 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1360/2727] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:01 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:01.946802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:01.958824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2494a-5c51-47c6-8fbd-d514b02629db service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.030005 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.030694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.030975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.031249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.031980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.033919 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.056912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1368/2728] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:01 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:02.062439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.078806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d88f7-cc58-45cc-9458-b475391053a6 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.084356 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.085461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.085762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.086048 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.087038 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.088784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.101678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1361/2729] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:02.108149 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.133319 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f3849-86a5-47c3-a5d3-e74443dc9dfa admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.142310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef97b71-4b36-4af4-8a63-068ebaf04bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.143104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97b71-4b36-4af4-8a63-068ebaf04bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.143402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97b71-4b36-4af4-8a63-068ebaf04bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.143705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97b71-4b36-4af4-8a63-068ebaf04bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.144454 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef97b71-4b36-4af4-8a63-068ebaf04bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.146066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef97b71-4b36-4af4-8a63-068ebaf04bf9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.158103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1369/2730] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:02.162894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.174819 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ab0b-a2e7-4e09-9838-8f9853b20d92 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.179917 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.180829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.181110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.181386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] PATH_INFO: `/v3/projects/ae48b7ac5cf94dc6b070996118c98bbd/users/b08f2f044a514ca7afb61f94f9dce902/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.197011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae48b7ac5cf94dc6b070996118c98bbd, user_id=b08f2f044a514ca7afb61f94f9dce902, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.199829 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.213352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1362/2731] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:02 2022] PUT /identity/v3/projects/ae48b7ac5cf94dc6b070996118c98bbd/users/b08f2f044a514ca7afb61f94f9dce902/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:02.224003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26204c-ed0c-4a9d-a0ad-0cdc57ef8642 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.224294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26204c-ed0c-4a9d-a0ad-0cdc57ef8642 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.224573 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26204c-ed0c-4a9d-a0ad-0cdc57ef8642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.253645 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-0a26204c-ed0c-4a9d-a0ad-0cdc57ef8642 None None] MFA Rules not processed for user `b08f2f044a514ca7afb61f94f9dce902`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:02.254863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0a26204c-ed0c-4a9d-a0ad-0cdc57ef8642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:02.327523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:02.331920 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1370/2732] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:02.336722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.411425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532d508-da9f-4aed-9121-9b11e906ea2f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.416424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.417104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.417389 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.417670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.418392 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.420417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.452465 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1363/2733] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:02 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:02.456000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26204c-ed0c-4a9d-a0ad-0cdc57ef8642 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.467824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26204c-ed0c-4a9d-a0ad-0cdc57ef8642 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.534914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.535628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.535908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.536181 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.536869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.538857 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.567164 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1371/2734] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:02 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:02.571152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.582166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a8a2-96ce-49f4-886f-d2f5a77ad620 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.587218 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.587966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.588247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.588528 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.589299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.591221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.602523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1364/2735] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:02.609221 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.623336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db81096-e156-4fe5-b02d-8cb6b896dd52 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.628858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.629815 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.630084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.630461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] PATH_INFO: `/v3/projects/5cce23b8d9b742cdb0e3df45a338e863/users/e6b2e90983a947569996d6560487c026/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.649385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cce23b8d9b742cdb0e3df45a338e863, user_id=e6b2e90983a947569996d6560487c026, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.652169 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.668385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1372/2736] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:02 2022] PUT /identity/v3/projects/5cce23b8d9b742cdb0e3df45a338e863/users/e6b2e90983a947569996d6560487c026/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:02.674110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.679784 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:02.685701 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac361de-0913-4c89-9bb6-4341898028aa admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.763760 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d616320-e085-456e-8a0f-db9930ef31fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.764460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616320-e085-456e-8a0f-db9930ef31fe admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.764785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616320-e085-456e-8a0f-db9930ef31fe admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.765034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616320-e085-456e-8a0f-db9930ef31fe admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.766618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d616320-e085-456e-8a0f-db9930ef31fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.768251 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d616320-e085-456e-8a0f-db9930ef31fe admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.783001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1365/2737] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:02.786515 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:02.803327 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ada5d-f1e7-4a98-aec1-143fcba6865b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:02.808870 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:02.810720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.811249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.811354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] PATH_INFO: `/v3/projects/5cce23b8d9b742cdb0e3df45a338e863/users/e6b2e90983a947569996d6560487c026/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.813604 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cce23b8d9b742cdb0e3df45a338e863, user_id=e6b2e90983a947569996d6560487c026, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:02.816522 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:02.830615 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1373/2738] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:02 2022] PUT /identity/v3/projects/5cce23b8d9b742cdb0e3df45a338e863/users/e6b2e90983a947569996d6560487c026/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:02.898780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f243157-fcaf-4da3-ae83-7833a85421d2 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:02.899067 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f243157-fcaf-4da3-ae83-7833a85421d2 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:02.899345 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f243157-fcaf-4da3-ae83-7833a85421d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:02.928328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-4f243157-fcaf-4da3-ae83-7833a85421d2 None None] MFA Rules not processed for user `e6b2e90983a947569996d6560487c026`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:02.929439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-4f243157-fcaf-4da3-ae83-7833a85421d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:03.000307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1366/2739] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:03.012459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:03.014727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:03.077535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e1a35-0ac3-4a22-b482-3f226e656c68 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:03.082336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:03.083198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:03.083541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:03.083853 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:03.093383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:03.094855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:03.104311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1374/2740] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:03.147036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f243157-fcaf-4da3-ae83-7833a85421d2 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:03.159268 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f243157-fcaf-4da3-ae83-7833a85421d2 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:03.163965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628331d-13de-4836-afea-69fcc1b3bdb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:03.164682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628331d-13de-4836-afea-69fcc1b3bdb3 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:03.164951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628331d-13de-4836-afea-69fcc1b3bdb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:03.165369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628331d-13de-4836-afea-69fcc1b3bdb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:03.225854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628331d-13de-4836-afea-69fcc1b3bdb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:03.227354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628331d-13de-4836-afea-69fcc1b3bdb3 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:03.238758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1367/2741] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:06.153427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:06.212237 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:06.219238 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b26801-3084-433e-a0a3-8cdcaae58246 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:06.224492 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:06.225366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:06.225675 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:06.225970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] PATH_INFO: `/v3/users/e8a615ea755d4224a7b91858262531e5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:06.227209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a615ea755d4224a7b91858262531e5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:06.229262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:06.282729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a615ea755d4224a7b91858262531e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.295673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a615ea755d4224a7b91858262531e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.303968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a615ea755d4224a7b91858262531e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.311792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a615ea755d4224a7b91858262531e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.312516 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a615ea755d4224a7b91858262531e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.318409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1375/2742] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:06 2022] DELETE /identity/v3/users/e8a615ea755d4224a7b91858262531e5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:06.323487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:06.324033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:06.324033 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:06.344011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:06.345262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:06.364915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:06.411586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1368/2743] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:06.421503 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:06.423160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:06.485267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f900e-31d8-4972-8a85-1d4eed1b7e6e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:06.490098 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:06.490770 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:06.491052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:06.491328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:06.501796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:06.503988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:06.514299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1376/2744] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:06.803059 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:06.804696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:06.818408 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827ede2-0d32-471f-ac58-bd90e082668e None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:06.823814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:06.824929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:06.825238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:06.825515 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] PATH_INFO: `/v3/projects/8e28e29f9679406aaf7038b4291e3106` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:06.826848 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e28e29f9679406aaf7038b4291e3106)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:06.828836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:06.878984 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e28e29f9679406aaf7038b4291e3106', 'request_id': 'req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.885513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e28e29f9679406aaf7038b4291e3106', 'request_id': 'req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.892466 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e28e29f9679406aaf7038b4291e3106', 'request_id': 'req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.907124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] The token cache is being invalidate because project 8e28e29f9679406aaf7038b4291e3106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:06.907441 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.908075 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:06.909643 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1369/2745] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:06 2022] DELETE /identity/v3/projects/8e28e29f9679406aaf7038b4291e3106 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:06.913993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:06.923923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a595dd-eee2-4045-be2a-71e56b1fe4e0 service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:06.984030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:06.984918 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:06.985204 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:06.985478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] PATH_INFO: `/v3/users/b08f2f044a514ca7afb61f94f9dce902` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:06.986679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08f2f044a514ca7afb61f94f9dce902)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:06.988637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:07.010717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:07.036954 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08f2f044a514ca7afb61f94f9dce902'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.049104 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08f2f044a514ca7afb61f94f9dce902'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.056695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08f2f044a514ca7afb61f94f9dce902'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.064602 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08f2f044a514ca7afb61f94f9dce902'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.065343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08f2f044a514ca7afb61f94f9dce902'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.072316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1377/2746] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:06 2022] DELETE /identity/v3/users/b08f2f044a514ca7afb61f94f9dce902 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:07.375457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:07.376961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:07.392619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44fb63-a1e7-46e5-a7ed-e718251e5e98 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:07.453785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:07.454657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:07.454942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:07.455220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] PATH_INFO: `/v3/projects/ae48b7ac5cf94dc6b070996118c98bbd` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:07.456487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae48b7ac5cf94dc6b070996118c98bbd)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:07.458415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:07.505852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:07.507601 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae48b7ac5cf94dc6b070996118c98bbd', 'request_id': 'req-50deb557-79ea-4154-9ed7-59a8ba5bf3de'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.514962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae48b7ac5cf94dc6b070996118c98bbd', 'request_id': 'req-50deb557-79ea-4154-9ed7-59a8ba5bf3de'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.521252 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae48b7ac5cf94dc6b070996118c98bbd', 'request_id': 'req-50deb557-79ea-4154-9ed7-59a8ba5bf3de'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.534290 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] The token cache is being invalidate because project ae48b7ac5cf94dc6b070996118c98bbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:07.534580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.535189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.536793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1370/2747] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:07 2022] DELETE /identity/v3/projects/ae48b7ac5cf94dc6b070996118c98bbd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:07.541133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:07.542601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:07.556998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae721c-67bc-454d-a401-fd250c12dc6e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:07.616676 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:07.617572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:07.617851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:07.618101 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] PATH_INFO: `/v3/users/e6b2e90983a947569996d6560487c026` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:07.619354 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b2e90983a947569996d6560487c026)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:07.621333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:07.670092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b2e90983a947569996d6560487c026'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.682758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b2e90983a947569996d6560487c026'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.690988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b2e90983a947569996d6560487c026'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.698021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b2e90983a947569996d6560487c026'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.698704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-469127f7-be4a-45e1-9aa6-518cccb8fb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b2e90983a947569996d6560487c026'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:07.705469 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1378/2748] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:07 2022] DELETE /identity/v3/users/e6b2e90983a947569996d6560487c026 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:07.999436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:07.999436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:08.010017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50deb557-79ea-4154-9ed7-59a8ba5bf3de admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.086030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.087503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.087871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.088217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] PATH_INFO: `/v3/projects/5cce23b8d9b742cdb0e3df45a338e863` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.090043 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cce23b8d9b742cdb0e3df45a338e863)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.092520 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.150771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cce23b8d9b742cdb0e3df45a338e863', 'request_id': 'req-fafbc0c2-6d8d-41d0-9188-df373d36f987'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:08.158236 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cce23b8d9b742cdb0e3df45a338e863', 'request_id': 'req-fafbc0c2-6d8d-41d0-9188-df373d36f987'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:08.164342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cce23b8d9b742cdb0e3df45a338e863', 'request_id': 'req-fafbc0c2-6d8d-41d0-9188-df373d36f987'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:08.180409 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] The token cache is being invalidate because project 5cce23b8d9b742cdb0e3df45a338e863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:08.180776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:08.181574 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-fafbc0c2-6d8d-41d0-9188-df373d36f987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:08.183890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1371/2749] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:07 2022] DELETE /identity/v3/projects/5cce23b8d9b742cdb0e3df45a338e863 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:08.190711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca900-1444-496c-b746-21ecd1e3751f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.191039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca900-1444-496c-b746-21ecd1e3751f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.191322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca900-1444-496c-b746-21ecd1e3751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.209431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-1bbca900-1444-496c-b746-21ecd1e3751f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:08.210648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-1bbca900-1444-496c-b746-21ecd1e3751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:08.220776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:08.273118 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1379/2750] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:08.281698 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854fef-adcd-4223-b6f0-9371f4a17062 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.281996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854fef-adcd-4223-b6f0-9371f4a17062 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.282283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854fef-adcd-4223-b6f0-9371f4a17062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.301068 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-2e854fef-adcd-4223-b6f0-9371f4a17062 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:08.302282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-2e854fef-adcd-4223-b6f0-9371f4a17062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:08.310863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1372/2751] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:08.315254 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbca900-1444-496c-b746-21ecd1e3751f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.327871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbca900-1444-496c-b746-21ecd1e3751f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.332863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.333540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.333830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.334108 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.334888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:08.335194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.337015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.351498 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1380/2752] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:08.356056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854fef-adcd-4223-b6f0-9371f4a17062 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.374800 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854fef-adcd-4223-b6f0-9371f4a17062 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.390104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.390883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.391877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.391877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.392230 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.394222 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.424279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1373/2753] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:08 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:08.428490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.439754 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812cece-74a8-4f16-bb21-4d0baf1aaf9f admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.508478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.509173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.509472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.509752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.510478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.512894 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.535927 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1381/2754] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:08 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:08.540357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.552446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bd6c9-45ef-408b-b4f6-d95fa19975fa admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.557665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.558348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.558805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.559207 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.560318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.561979 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.578535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1374/2755] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:08.583531 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.597322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2a49f-5277-491a-a204-52b1eecd016a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.602662 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5293bf-6755-47d9-9657-81e53882cb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.603410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5293bf-6755-47d9-9657-81e53882cb37 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.603710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5293bf-6755-47d9-9657-81e53882cb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.603978 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5293bf-6755-47d9-9657-81e53882cb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.604869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5293bf-6755-47d9-9657-81e53882cb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.606666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5293bf-6755-47d9-9657-81e53882cb37 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.621422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1382/2756] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:08.626350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.638894 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548684f-c358-41b6-88c1-acc401343406 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.644321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.645254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.645542 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.645822 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] PATH_INFO: `/v3/projects/659a7faea91b4f6eb8539669b04788b2/users/0b587b4c205b4facb9f7e58c1b5d6920/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.662609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=659a7faea91b4f6eb8539669b04788b2, user_id=0b587b4c205b4facb9f7e58c1b5d6920, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.665316 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.679908 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1375/2757] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:08 2022] PUT /identity/v3/projects/659a7faea91b4f6eb8539669b04788b2/users/0b587b4c205b4facb9f7e58c1b5d6920/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:08.692648 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8615a25-b38d-45d9-9b0b-d86197524bc3 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.693089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8615a25-b38d-45d9-9b0b-d86197524bc3 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.693376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8615a25-b38d-45d9-9b0b-d86197524bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.721612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-a8615a25-b38d-45d9-9b0b-d86197524bc3 None None] MFA Rules not processed for user `0b587b4c205b4facb9f7e58c1b5d6920`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:08.722856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-a8615a25-b38d-45d9-9b0b-d86197524bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:08.797561 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1383/2758] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:08.802457 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.807234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:08.869813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc6bb-cae6-4e2e-aa3a-3f27985111ab admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.874741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.875404 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.875741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.876019 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.876727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.878664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:08.905576 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1376/2759] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:08 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:08.909890 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8615a25-b38d-45d9-9b0b-d86197524bc3 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:08.921292 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8615a25-b38d-45d9-9b0b-d86197524bc3 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:08.930491 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:08.982751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:08.983463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:08.983747 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:08.984086 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:08.984796 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:08.986722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.008142 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1384/2760] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:08 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:09.012634 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.023000 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b316b6-dfdb-4d91-8466-d1a10d081ded admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.028084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.028828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.029115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.029431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.030148 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:09.031791 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.042640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1377/2761] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:09.047459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.057687 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78231eee-6def-45e6-9caf-0bc3027851b0 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.062784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.063717 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.064001 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.064242 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] PATH_INFO: `/v3/projects/bd7b27fbe7e943bd8e721199566bd796/users/7ba0a27e9ae5470b934502ecde5a594d/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.080487 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7b27fbe7e943bd8e721199566bd796, user_id=7ba0a27e9ae5470b934502ecde5a594d, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:09.083280 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.096479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1385/2762] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:09 2022] PUT /identity/v3/projects/bd7b27fbe7e943bd8e721199566bd796/users/7ba0a27e9ae5470b934502ecde5a594d/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:09.100606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.110956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c926-5cfd-4fb6-b9e1-d5130c742c51 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.170664 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69c121-159c-4017-a256-7469849b6554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.172217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69c121-159c-4017-a256-7469849b6554 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.172600 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69c121-159c-4017-a256-7469849b6554 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.172998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69c121-159c-4017-a256-7469849b6554 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.174271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69c121-159c-4017-a256-7469849b6554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:09.176620 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69c121-159c-4017-a256-7469849b6554 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.190231 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1378/2763] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:09.194794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.206359 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30dc778-349e-4be6-9416-d5a1160900f4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.212192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.213100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.213387 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.213667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] PATH_INFO: `/v3/projects/bd7b27fbe7e943bd8e721199566bd796/users/7ba0a27e9ae5470b934502ecde5a594d/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.215994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7b27fbe7e943bd8e721199566bd796, user_id=7ba0a27e9ae5470b934502ecde5a594d, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:09.218666 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.229338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1386/2764] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:09 2022] PUT /identity/v3/projects/bd7b27fbe7e943bd8e721199566bd796/users/7ba0a27e9ae5470b934502ecde5a594d/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:09.240614 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.240910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.241194 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.269932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] MFA Rules not processed for user `7ba0a27e9ae5470b934502ecde5a594d`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:09.271169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:09.340977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:09.348777 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1379/2765] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:09.359908 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.361779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:09.425641 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d59d5-5664-4b98-a524-cef184942405 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.429493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b14eb-52d3-4994-ba10-49411e1c223d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.430141 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b14eb-52d3-4994-ba10-49411e1c223d service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.430445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b14eb-52d3-4994-ba10-49411e1c223d service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.430700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b14eb-52d3-4994-ba10-49411e1c223d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.441608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b14eb-52d3-4994-ba10-49411e1c223d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:09.443334 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b14eb-52d3-4994-ba10-49411e1c223d service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.454081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1387/2766] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:09.544352 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.545801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:09.628042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37a210-f2ab-4821-8a2f-a246cd5132b6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.638317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.639229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.639580 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.639855 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] PATH_INFO: `/v3/users/c5b7e36490ce4b889de3edda9d22d9f8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.641100 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b7e36490ce4b889de3edda9d22d9f8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:09.643035 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.713824 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b7e36490ce4b889de3edda9d22d9f8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:09.738815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b7e36490ce4b889de3edda9d22d9f8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:09.752179 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b7e36490ce4b889de3edda9d22d9f8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:09.766890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b7e36490ce4b889de3edda9d22d9f8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:09.767713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b7e36490ce4b889de3edda9d22d9f8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:09.780152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1380/2767] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:09 2022] DELETE /identity/v3/users/c5b7e36490ce4b889de3edda9d22d9f8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:09.788410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009674aa-8147-40df-8920-30a12d517f0d None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.788410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009674aa-8147-40df-8920-30a12d517f0d None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.788410 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009674aa-8147-40df-8920-30a12d517f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.805033 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-009674aa-8147-40df-8920-30a12d517f0d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:09.806263 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-009674aa-8147-40df-8920-30a12d517f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:09.871055 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.873120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:09.898455 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1388/2768] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:09.908431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009674aa-8147-40df-8920-30a12d517f0d None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:09.909816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-009674aa-8147-40df-8920-30a12d517f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:09.922636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:09.961175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d7c4-4feb-4b40-9419-09b5639a1352 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.966241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.967124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.967437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.967722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] PATH_INFO: `/v3/users/0b587b4c205b4facb9f7e58c1b5d6920` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:09.969094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b587b4c205b4facb9f7e58c1b5d6920)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:09.970991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:09.983358 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009674aa-8147-40df-8920-30a12d517f0d None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:09.987673 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4382e-7438-45d6-9f7d-4b9ac3a7bc1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:09.988357 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4382e-7438-45d6-9f7d-4b9ac3a7bc1b service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:09.988646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4382e-7438-45d6-9f7d-4b9ac3a7bc1b service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:09.988922 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4382e-7438-45d6-9f7d-4b9ac3a7bc1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.001459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4382e-7438-45d6-9f7d-4b9ac3a7bc1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:10.001631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:10.001708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4382e-7438-45d6-9f7d-4b9ac3a7bc1b service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:10.012854 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1389/2769] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:10.030262 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b587b4c205b4facb9f7e58c1b5d6920'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.042964 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b587b4c205b4facb9f7e58c1b5d6920'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.050876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b587b4c205b4facb9f7e58c1b5d6920'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.059743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b587b4c205b4facb9f7e58c1b5d6920'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.060550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b587b4c205b4facb9f7e58c1b5d6920'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.068813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1381/2770] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:09 2022] DELETE /identity/v3/users/0b587b4c205b4facb9f7e58c1b5d6920 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:10.073131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:10.073432 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:10.073710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.093742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:10.095551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:10.192956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1390/2771] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:10.206816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:10.208293 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:10.266889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fac10-78f9-4325-8a8b-79ab2fc60b57 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:10.270850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:10.271534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:10.271816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:10.272087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.281493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:10.282969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:10.293042 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1382/2772] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:10.326171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:10.327657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:10.341060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352ef2f-19f2-432d-90ef-468d0626b2aa None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:10.346088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:10.347497 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:10.347731 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:10.348092 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] PATH_INFO: `/v3/projects/cb08466c0bed478785a2330ac19477e1` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.349623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb08466c0bed478785a2330ac19477e1)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:10.351679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:10.404816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb08466c0bed478785a2330ac19477e1', 'request_id': 'req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.412272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb08466c0bed478785a2330ac19477e1', 'request_id': 'req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.418079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb08466c0bed478785a2330ac19477e1', 'request_id': 'req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.433246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] The token cache is being invalidate because project cb08466c0bed478785a2330ac19477e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:10.433713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.434478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.436355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1391/2773] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:10 2022] DELETE /identity/v3/projects/cb08466c0bed478785a2330ac19477e1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:10.441131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:10.452449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d9506-484c-4868-ad04-6071a362e4d2 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:10.523073 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:10.524003 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:10.524287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:10.524564 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] PATH_INFO: `/v3/users/5397622ec534492dadbf95a786765b3f` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.525785 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5397622ec534492dadbf95a786765b3f)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:10.527849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:10.573625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:10.580700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5397622ec534492dadbf95a786765b3f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.594573 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5397622ec534492dadbf95a786765b3f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.601949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:10.602648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5397622ec534492dadbf95a786765b3f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.603548 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:10.614223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5397622ec534492dadbf95a786765b3f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.615006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5397622ec534492dadbf95a786765b3f'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.619592 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a5b4-bbf1-4670-83cd-b3e7ce78893d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:10.622057 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1383/2774] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:10 2022] DELETE /identity/v3/users/5397622ec534492dadbf95a786765b3f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:10.627130 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:10.627606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:10.627888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:10.628160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] PATH_INFO: `/v3/projects/659a7faea91b4f6eb8539669b04788b2` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.629482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] RBAC: Authorizing `identity:delete_project(project_id=659a7faea91b4f6eb8539669b04788b2)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:10.631461 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:10.669010 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:10.688459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659a7faea91b4f6eb8539669b04788b2', 'request_id': 'req-495b4ccb-96ac-45d8-8a81-2a25c19fabff'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.695889 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659a7faea91b4f6eb8539669b04788b2', 'request_id': 'req-495b4ccb-96ac-45d8-8a81-2a25c19fabff'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.702521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659a7faea91b4f6eb8539669b04788b2', 'request_id': 'req-495b4ccb-96ac-45d8-8a81-2a25c19fabff'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.715009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] The token cache is being invalidate because project 659a7faea91b4f6eb8539669b04788b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:10.715489 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.716257 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.718089 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1392/2775] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:10 2022] DELETE /identity/v3/projects/659a7faea91b4f6eb8539669b04788b2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:10.721977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:10.725330 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:10.798040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4002ac-a166-4929-a307-86912f019a02 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:10.854298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:10.855234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:10.855549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:10.855876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] PATH_INFO: `/v3/users/7ba0a27e9ae5470b934502ecde5a594d` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.857449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba0a27e9ae5470b934502ecde5a594d)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:10.859395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:10.913419 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:10.914996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba0a27e9ae5470b934502ecde5a594d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.915134 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:10.929486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b4ccb-96ac-45d8-8a81-2a25c19fabff admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:10.929946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba0a27e9ae5470b934502ecde5a594d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.932388 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:10.935395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:10.936521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:10.936814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:10.937160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] PATH_INFO: `/v3/projects/387127c805ca4aa593f0c37be0c6ff58` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:10.938642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] RBAC: Authorizing `identity:delete_project(project_id=387127c805ca4aa593f0c37be0c6ff58)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:10.939321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba0a27e9ae5470b934502ecde5a594d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.940899 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:10.951170 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba0a27e9ae5470b934502ecde5a594d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.951899 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cd3d885e-e58c-4df6-b291-9d6c832c0e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba0a27e9ae5470b934502ecde5a594d'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:10.959001 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1384/2776] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:10 2022] DELETE /identity/v3/users/7ba0a27e9ae5470b934502ecde5a594d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:10.996627 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387127c805ca4aa593f0c37be0c6ff58', 'request_id': 'req-946f2b0c-bccf-403d-b3e0-4787bf22800c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.004684 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387127c805ca4aa593f0c37be0c6ff58', 'request_id': 'req-946f2b0c-bccf-403d-b3e0-4787bf22800c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.013654 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387127c805ca4aa593f0c37be0c6ff58', 'request_id': 'req-946f2b0c-bccf-403d-b3e0-4787bf22800c'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.036483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] The token cache is being invalidate because project 387127c805ca4aa593f0c37be0c6ff58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:11.036885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.037748 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-946f2b0c-bccf-403d-b3e0-4787bf22800c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.039855 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1393/2777] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:10 2022] DELETE /identity/v3/projects/387127c805ca4aa593f0c37be0c6ff58 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:11.045325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac50e1-080b-46c5-b018-d191dc9089a0 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.045696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac50e1-080b-46c5-b018-d191dc9089a0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.046116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac50e1-080b-46c5-b018-d191dc9089a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.067434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-57ac50e1-080b-46c5-b018-d191dc9089a0 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:11.069024 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-57ac50e1-080b-46c5-b018-d191dc9089a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:11.138377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1385/2778] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:11.146679 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f681-8af5-4c9a-a4a6-65a9fce058fc None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.146977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f681-8af5-4c9a-a4a6-65a9fce058fc None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.147264 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f681-8af5-4c9a-a4a6-65a9fce058fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.171655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-dc59f681-8af5-4c9a-a4a6-65a9fce058fc None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:11.173545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-dc59f681-8af5-4c9a-a4a6-65a9fce058fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:11.187114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1394/2779] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:11.188204 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac50e1-080b-46c5-b018-d191dc9089a0 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.201116 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac50e1-080b-46c5-b018-d191dc9089a0 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.206168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.207069 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.207354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.207657 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.208439 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:11.208723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.210444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.224832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1386/2780] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:11.229234 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f681-8af5-4c9a-a4a6-65a9fce058fc None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.245482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f681-8af5-4c9a-a4a6-65a9fce058fc None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.250752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.251773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.252063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.252337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.253049 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.254995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.299027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1395/2781] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/projects => generated 391 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:11.303192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.316981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d0e6-b927-4b03-b09b-ab4b1e32cf2c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.384467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.385173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.385472 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.385744 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.386473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.389040 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.414353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1387/2782] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:11.415668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.426761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75737e10-f00d-4a64-a7cb-231fcaf8f42a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.432391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.433087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.433374 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.433658 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.434828 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.436605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.451395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1396/2783] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:11.452645 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.463794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d63b84-56c3-4583-9475-c20cf083a1c1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.469010 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94aa2b-dcc5-4fcf-b7f7-7a2b013be789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.469700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94aa2b-dcc5-4fcf-b7f7-7a2b013be789 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.469985 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94aa2b-dcc5-4fcf-b7f7-7a2b013be789 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.470264 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94aa2b-dcc5-4fcf-b7f7-7a2b013be789 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.471036 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94aa2b-dcc5-4fcf-b7f7-7a2b013be789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.472759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94aa2b-dcc5-4fcf-b7f7-7a2b013be789 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.487177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1388/2784] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:11.488508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.499534 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246c5f8-58c2-4830-827a-a8fb4ab55d2a admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.504580 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.505559 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.505912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.506285 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] PATH_INFO: `/v3/projects/0cc74f7624cd49658f030d963d8f5d75/users/956a97369567406da4d743fd091938c8/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.522382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc74f7624cd49658f030d963d8f5d75, user_id=956a97369567406da4d743fd091938c8, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.525232 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.539842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1397/2785] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:11 2022] PUT /identity/v3/projects/0cc74f7624cd49658f030d963d8f5d75/users/956a97369567406da4d743fd091938c8/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:11.552568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82a66a-db6b-4680-8156-b8cbf76159f4 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.552960 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82a66a-db6b-4680-8156-b8cbf76159f4 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.553273 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82a66a-db6b-4680-8156-b8cbf76159f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.560762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.562757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:11.590411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-3d82a66a-db6b-4680-8156-b8cbf76159f4 None None] MFA Rules not processed for user `956a97369567406da4d743fd091938c8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:11.592012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-3d82a66a-db6b-4680-8156-b8cbf76159f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:11.613943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:11.635610 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:11.641545 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd37c01-5e09-4a03-b73f-d7217e476262 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.647530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.648630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.648915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.649192 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] PATH_INFO: `/v3/projects/bd7b27fbe7e943bd8e721199566bd796` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.650472 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7b27fbe7e943bd8e721199566bd796)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.652442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.663166 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1389/2786] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:11.667449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82a66a-db6b-4680-8156-b8cbf76159f4 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.677951 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82a66a-db6b-4680-8156-b8cbf76159f4 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.683401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.684305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.684719 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.685119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.685935 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.687976 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.702121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7b27fbe7e943bd8e721199566bd796', 'request_id': 'req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.709028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7b27fbe7e943bd8e721199566bd796', 'request_id': 'req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.714139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1390/2787] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:11.715074 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7b27fbe7e943bd8e721199566bd796', 'request_id': 'req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.718150 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.728986 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] The token cache is being invalidate because project bd7b27fbe7e943bd8e721199566bd796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:11.729284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.729987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-cddf5414-b7a7-4f68-a9e8-fd195da4d35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:11.732088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1398/2788] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:11 2022] DELETE /identity/v3/projects/bd7b27fbe7e943bd8e721199566bd796 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:11.736882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6852133-a0e3-48a0-823a-585643304912 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.737178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6852133-a0e3-48a0-823a-585643304912 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.737462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6852133-a0e3-48a0-823a-585643304912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.755538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c6852133-a0e3-48a0-823a-585643304912 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:11.757044 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c6852133-a0e3-48a0-823a-585643304912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:11.784718 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ddfca-9d67-4711-93dd-6b392d6f6cdd admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.789583 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de24f95-a781-46df-9b2b-b26163881bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.790265 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de24f95-a781-46df-9b2b-b26163881bcb admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.790586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de24f95-a781-46df-9b2b-b26163881bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.790837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de24f95-a781-46df-9b2b-b26163881bcb admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.791567 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de24f95-a781-46df-9b2b-b26163881bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.793586 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de24f95-a781-46df-9b2b-b26163881bcb admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.816197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1391/2789] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:11.817961 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1399/2790] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:11.820112 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6852133-a0e3-48a0-823a-585643304912 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.821460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c6852133-a0e3-48a0-823a-585643304912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:11.826134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf627-adfe-4fb7-8080-7f09add81fa6 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.826431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf627-adfe-4fb7-8080-7f09add81fa6 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.826714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf627-adfe-4fb7-8080-7f09add81fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.833937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6852133-a0e3-48a0-823a-585643304912 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.839529 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.840218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.840500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.840775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.841541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.843163 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.845727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-11ccf627-adfe-4fb7-8080-7f09add81fa6 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:11.846975 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-11ccf627-adfe-4fb7-8080-7f09add81fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:11.853657 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1400/2791] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:11.855017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1392/2792] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:11.858542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.861437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf627-adfe-4fb7-8080-7f09add81fa6 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.868446 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7d0aa-5b27-45b4-8e68-6b1feffa3021 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.870298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf627-adfe-4fb7-8080-7f09add81fa6 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.873647 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.874538 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.874941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.875224 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] PATH_INFO: `/v3/projects/e140b54b55af4f3a94c50f071081a110/users/5a1cf324c0614c5b9d6172ffa631bacf/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.875400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.876065 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.876351 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.876628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.877369 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:11.877680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.879297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.892012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1393/2793] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 13:29:11.895631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e140b54b55af4f3a94c50f071081a110, user_id=5a1cf324c0614c5b9d6172ffa631bacf, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.896427 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.898366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.906950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b1f66-5eae-44bd-acf0-f348b7dbef70 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.912276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1401/2794] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:11 2022] PUT /identity/v3/projects/e140b54b55af4f3a94c50f071081a110/users/5a1cf324c0614c5b9d6172ffa631bacf/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:11.916678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:11.952445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:11.967607 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.968233 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.968592 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.968970 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.969738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.971805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:11.988028 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a102dd2-5c95-463c-a0ab-04199c2151b2 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:11.994122 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:11.994809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:11.995087 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:11.995363 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:11.996129 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:11.997814 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.004101 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1394/2795] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:11 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:12.005130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.008714 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1402/2796] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:12.013557 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.022808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:12.023395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d97ae-2de6-44d4-9e24-9a3381a7a6f6 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.076096 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e896b3c-5875-4b91-b785-667d1344978a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.081280 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a9cc-3690-42eb-aef2-0180e53671c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.081958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a9cc-3690-42eb-aef2-0180e53671c2 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.082238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a9cc-3690-42eb-aef2-0180e53671c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.082513 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a9cc-3690-42eb-aef2-0180e53671c2 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.083071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.083191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a9cc-3690-42eb-aef2-0180e53671c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.083998 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.084282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.084734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] PATH_INFO: `/v3/projects/e140b54b55af4f3a94c50f071081a110/users/5a1cf324c0614c5b9d6172ffa631bacf/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.085254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a9cc-3690-42eb-aef2-0180e53671c2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.086635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e140b54b55af4f3a94c50f071081a110, user_id=5a1cf324c0614c5b9d6172ffa631bacf, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.089337 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.101728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1403/2797] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:12 2022] PUT /identity/v3/projects/e140b54b55af4f3a94c50f071081a110/users/5a1cf324c0614c5b9d6172ffa631bacf/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:12.107738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1395/2798] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:12 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:12.112277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.113070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b44e-087b-47bc-b860-c39379af047b None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.113376 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b44e-087b-47bc-b860-c39379af047b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.113663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b44e-087b-47bc-b860-c39379af047b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.142939 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9418b44e-087b-47bc-b860-c39379af047b None None] MFA Rules not processed for user `5a1cf324c0614c5b9d6172ffa631bacf`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:12.144169 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9418b44e-087b-47bc-b860-c39379af047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:12.180902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2c9e7-2430-4757-abf8-9da5ef88f6fa admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.185742 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.186422 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.186704 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.186985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.187761 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.189384 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.199987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1404/2799] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:12.204143 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.214326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb1150-a8c5-4df4-9f36-6433a348a7a1 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.216402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1396/2800] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:12.223335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.224065 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.224347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.224623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.225369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.227055 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.227401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418b44e-087b-47bc-b860-c39379af047b None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.228803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9418b44e-087b-47bc-b860-c39379af047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:12.239050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1405/2801] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:12.243424 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.254210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ac28-162f-4035-ba81-959402ded5ee admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.258931 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd825b-f335-40c3-8fd6-0f6724c0ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.259876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd825b-f335-40c3-8fd6-0f6724c0ff5e admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.260155 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd825b-f335-40c3-8fd6-0f6724c0ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.260433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd825b-f335-40c3-8fd6-0f6724c0ff5e admin admin] PATH_INFO: `/v3/projects/97d4b3991d344934aaced8ebc4720694/users/438f04b06f984769a187ee4fd546b2ab/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.276842 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd825b-f335-40c3-8fd6-0f6724c0ff5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d4b3991d344934aaced8ebc4720694, user_id=438f04b06f984769a187ee4fd546b2ab, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.279763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd825b-f335-40c3-8fd6-0f6724c0ff5e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.290667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418b44e-087b-47bc-b860-c39379af047b None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.295146 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1406/2802] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:12 2022] PUT /identity/v3/projects/97d4b3991d344934aaced8ebc4720694/users/438f04b06f984769a187ee4fd546b2ab/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:12.307013 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ccbf5-65aa-4dfd-8d39-43d9f86630a6 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.307399 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ccbf5-65aa-4dfd-8d39-43d9f86630a6 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.307688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ccbf5-65aa-4dfd-8d39-43d9f86630a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.334383 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:12.335970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-9b8ccbf5-65aa-4dfd-8d39-43d9f86630a6 None None] MFA Rules not processed for user `438f04b06f984769a187ee4fd546b2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:12.337223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-9b8ccbf5-65aa-4dfd-8d39-43d9f86630a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:12.344591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.345750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.346229 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.346922 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.349276 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:12.424093 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1407/2803] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:12.428555 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ccbf5-65aa-4dfd-8d39-43d9f86630a6 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.429104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.430748 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.443286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1397/2804] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:12.498401 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.500933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ccbf5-65aa-4dfd-8d39-43d9f86630a6 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.507431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526a828-4f80-4038-a761-8f8886494577 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.507638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.508388 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.508727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.509060 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.509881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.511052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.511746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.512929 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.513207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.513304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.514723 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:12.545801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1408/2805] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:12 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:12.550167 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.585339 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.586813 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.597129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1398/2806] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:12.629077 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50453-ace9-4c8c-b1d9-7d70ba750f23 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.634187 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.634879 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.635296 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.635677 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.636524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.638926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.669069 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1409/2807] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:12 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:12.673183 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.684915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec77972-f64f-4e5e-9bf1-f78876588806 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.691775 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.692540 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.692930 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.693270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.694077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.695832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.710867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1399/2808] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:12.711982 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.722937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3263-056c-4ac0-ad04-09d7fa6a66b9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.728476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.729620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.729720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.729942 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] PATH_INFO: `/v3/projects/8f3564584a844442884f53c8c155c1fb/users/11e428714b1e4f52abafafce3f1a7c73/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.749258 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3564584a844442884f53c8c155c1fb, user_id=11e428714b1e4f52abafafce3f1a7c73, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.752100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.767706 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1410/2809] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:12 2022] PUT /identity/v3/projects/8f3564584a844442884f53c8c155c1fb/users/11e428714b1e4f52abafafce3f1a7c73/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:12.771933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.794555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508de47-05aa-4475-897d-a944260b6c5c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.872764 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54694339-490b-4924-934c-709e4eb5dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.873588 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694339-490b-4924-934c-709e4eb5dfeb admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.873974 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694339-490b-4924-934c-709e4eb5dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.874379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694339-490b-4924-934c-709e4eb5dfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.875232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694339-490b-4924-934c-709e4eb5dfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.878335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694339-490b-4924-934c-709e4eb5dfeb admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.895406 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1400/2810] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:12.898355 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:12.907035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:12.918350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1d46-f9f9-4c30-9efb-168bda6106c0 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:12.933283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:12.934488 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.935040 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.935520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] PATH_INFO: `/v3/projects/8f3564584a844442884f53c8c155c1fb/users/11e428714b1e4f52abafafce3f1a7c73/roles/4a6b9a3327544100929540645a797d97` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:12.938501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3564584a844442884f53c8c155c1fb, user_id=11e428714b1e4f52abafafce3f1a7c73, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:12.941782 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:12.967951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1411/2811] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:12 2022] PUT /identity/v3/projects/8f3564584a844442884f53c8c155c1fb/users/11e428714b1e4f52abafafce3f1a7c73/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:12.970522 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb55a36-48ba-4232-8bae-22c027c23af2 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:12.970814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb55a36-48ba-4232-8bae-22c027c23af2 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:12.971157 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb55a36-48ba-4232-8bae-22c027c23af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:13.005628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-deb55a36-48ba-4232-8bae-22c027c23af2 None None] MFA Rules not processed for user `11e428714b1e4f52abafafce3f1a7c73`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:13.006923 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-deb55a36-48ba-4232-8bae-22c027c23af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:13.012969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=100 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:13.081291 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1401/2812] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:13.093670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:13.095394 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:13.157007 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e435ab4-a582-4039-a717-3b687d54d927 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:13.160650 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:13.161298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:13.161551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:13.161824 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:13.172284 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:13.174170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:13.184891 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1412/2813] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:13.271131 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb55a36-48ba-4232-8bae-22c027c23af2 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:13.282223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb55a36-48ba-4232-8bae-22c027c23af2 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:13.285849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a277a93-2e97-434f-adf7-d10c796b112f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:13.286489 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277a93-2e97-434f-adf7-d10c796b112f service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:13.286773 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277a93-2e97-434f-adf7-d10c796b112f service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:13.287046 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277a93-2e97-434f-adf7-d10c796b112f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:13.345921 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a277a93-2e97-434f-adf7-d10c796b112f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:13.347468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a277a93-2e97-434f-adf7-d10c796b112f service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:13.359686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1402/2814] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:13.632501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:13.697834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49647c-8591-4004-a74c-7ddac3f0d38e service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:13.703071 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:13.703994 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:13.704262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:13.704552 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] PATH_INFO: `/v3/users/438f04b06f984769a187ee4fd546b2ab` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:13.705757 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438f04b06f984769a187ee4fd546b2ab)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:13.707720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:13.755205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438f04b06f984769a187ee4fd546b2ab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:13.767535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438f04b06f984769a187ee4fd546b2ab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:13.775668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438f04b06f984769a187ee4fd546b2ab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:13.783231 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438f04b06f984769a187ee4fd546b2ab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:13.783907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438f04b06f984769a187ee4fd546b2ab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:13.789885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1413/2815] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:13 2022] DELETE /identity/v3/users/438f04b06f984769a187ee4fd546b2ab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:13.794687 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:13.795182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:13.795182 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:13.812655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:13.813868 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:13.875317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1403/2816] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:13.885310 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:13.886904 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:13.901614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:13.950031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c505d-a198-4a85-aa76-9365c2900fd1 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:13.954160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:13.954830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:13.955111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:13.955730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:13.965995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:13.967532 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:13.977830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1414/2817] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:14.449245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:14.451398 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:14.464525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310378aa-38ff-469e-9235-f3d2d6e2d73d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:14.469363 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:14.470397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:14.470911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:14.471292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] PATH_INFO: `/v3/projects/97d4b3991d344934aaced8ebc4720694` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:14.472646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d4b3991d344934aaced8ebc4720694)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:14.474684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:14.547839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:14.559227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d4b3991d344934aaced8ebc4720694', 'request_id': 'req-e44d84d6-a637-405a-9b8f-a9e631bb45bc'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.569146 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d4b3991d344934aaced8ebc4720694', 'request_id': 'req-e44d84d6-a637-405a-9b8f-a9e631bb45bc'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.575909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d4b3991d344934aaced8ebc4720694', 'request_id': 'req-e44d84d6-a637-405a-9b8f-a9e631bb45bc'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.592782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] The token cache is being invalidate because project 97d4b3991d344934aaced8ebc4720694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:14.593301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.594012 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.599319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1404/2818] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:14 2022] DELETE /identity/v3/projects/97d4b3991d344934aaced8ebc4720694 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:14.600267 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:14.611038 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c973c-ded0-46a4-bbcb-2b5fdffe821d service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:14.673823 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:14.674933 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:14.675154 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:14.675636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] PATH_INFO: `/v3/users/11e428714b1e4f52abafafce3f1a7c73` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:14.677433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e428714b1e4f52abafafce3f1a7c73)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:14.683691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:14.741525 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e428714b1e4f52abafafce3f1a7c73'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.757777 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e428714b1e4f52abafafce3f1a7c73'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.767597 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e428714b1e4f52abafafce3f1a7c73'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.775809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e428714b1e4f52abafafce3f1a7c73'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.777097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-221b5b87-fa78-4bc7-bf0e-bac9b86b6a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e428714b1e4f52abafafce3f1a7c73'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:14.784556 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1415/2819] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:14 2022] DELETE /identity/v3/users/11e428714b1e4f52abafafce3f1a7c73 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:15.213387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.215141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.243841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d84d6-a637-405a-9b8f-a9e631bb45bc admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.313523 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.314606 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.314876 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.315155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] PATH_INFO: `/v3/projects/8f3564584a844442884f53c8c155c1fb` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.316402 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f3564584a844442884f53c8c155c1fb)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.318336 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.371881 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f3564584a844442884f53c8c155c1fb', 'request_id': 'req-38742fa8-8cc5-4921-beee-2bf563a3d2e6'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.378178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f3564584a844442884f53c8c155c1fb', 'request_id': 'req-38742fa8-8cc5-4921-beee-2bf563a3d2e6'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.383858 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f3564584a844442884f53c8c155c1fb', 'request_id': 'req-38742fa8-8cc5-4921-beee-2bf563a3d2e6'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.398358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] The token cache is being invalidate because project 8f3564584a844442884f53c8c155c1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:15.398654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.399286 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-38742fa8-8cc5-4921-beee-2bf563a3d2e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.401135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1405/2820] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:15 2022] DELETE /identity/v3/projects/8f3564584a844442884f53c8c155c1fb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:15.406201 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.406691 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.406775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.426772 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:15.428138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.460629 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:15.505744 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1416/2821] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:15.507811 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08c0e-141f-49ec-8c57-d942ad6b31d9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.508104 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08c0e-141f-49ec-8c57-d942ad6b31d9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.508384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08c0e-141f-49ec-8c57-d942ad6b31d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.512229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.514305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.529868 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e0d6e-2219-4bf7-9e34-921b6e1575e8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.531081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-68d08c0e-141f-49ec-8c57-d942ad6b31d9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:15.532425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-68d08c0e-141f-49ec-8c57-d942ad6b31d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.534758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.535671 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.535952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.536273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] PATH_INFO: `/v3/users/aca063167cd64b71b422e24121b13434` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.537457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca063167cd64b71b422e24121b13434)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.539435 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.540714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1406/2822] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:15.545187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d08c0e-141f-49ec-8c57-d942ad6b31d9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.554400 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d08c0e-141f-49ec-8c57-d942ad6b31d9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.559299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.560056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.560331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.560581 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.561317 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:15.561619 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.563224 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.579205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1407/2823] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 13:29:15.580306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.590213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca063167cd64b71b422e24121b13434'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.590693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1be96-011c-4dc2-805f-4e294f915c0c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.596948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.597609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.597898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.598174 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.598875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.600814 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.603937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca063167cd64b71b422e24121b13434'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.612793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca063167cd64b71b422e24121b13434'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.621685 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca063167cd64b71b422e24121b13434'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.622452 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca063167cd64b71b422e24121b13434'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.628133 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1408/2824] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:15 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:15.632026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1417/2825] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:15 2022] DELETE /identity/v3/users/aca063167cd64b71b422e24121b13434 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:15.632308 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.633883 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.648663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:15.700999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311624d-f022-4d25-8552-c17eae1f8d7e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.758027 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.758699 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.758981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.759254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.759999 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.761967 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.785149 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1409/2826] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:15 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:15.789272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.799865 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def33d0a-0634-439c-8ad8-0328c0246401 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.804965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.805640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.805907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.806198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.807036 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.808800 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.819458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1418/2827] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:15.824119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.833229 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.834969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.835487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e67c-2538-481f-8122-f919d36d5621 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.848100 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.848834 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f151f5f-ecbf-461f-b6b4-5e52790eb86c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.848942 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.849124 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.849403 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.850155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.851830 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.854048 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.855066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.855535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.855993 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] PATH_INFO: `/v3/projects/01a98b3c25264b86ae1e016303015650` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.857999 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a98b3c25264b86ae1e016303015650)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.860114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.864547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1410/2828] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:15.868716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.879192 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f49bdc-3e90-464b-a545-d58db3513d94 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:15.884898 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c34958-0d83-4e79-830d-4fdbebbd5650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:15.886460 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34958-0d83-4e79-830d-4fdbebbd5650 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.887178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34958-0d83-4e79-830d-4fdbebbd5650 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.887665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34958-0d83-4e79-830d-4fdbebbd5650 admin admin] PATH_INFO: `/v3/projects/9f83838c08d142038c759ee8d51a9262/users/312fba0e9b894db290fb2d48d0d704e9/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.904517 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c34958-0d83-4e79-830d-4fdbebbd5650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f83838c08d142038c759ee8d51a9262, user_id=312fba0e9b894db290fb2d48d0d704e9, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:15.907242 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c34958-0d83-4e79-830d-4fdbebbd5650 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:15.916809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a98b3c25264b86ae1e016303015650', 'request_id': 'req-4c6af646-8784-41b7-ab49-0bf6b252711b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.924369 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a98b3c25264b86ae1e016303015650', 'request_id': 'req-4c6af646-8784-41b7-ab49-0bf6b252711b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.925768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1411/2829] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:15 2022] PUT /identity/v3/projects/9f83838c08d142038c759ee8d51a9262/users/312fba0e9b894db290fb2d48d0d704e9/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:15.932863 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a98b3c25264b86ae1e016303015650', 'request_id': 'req-4c6af646-8784-41b7-ab49-0bf6b252711b'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.938081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:15.938482 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:15.938801 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:15.951054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] The token cache is being invalidate because project 01a98b3c25264b86ae1e016303015650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:15.951054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.951054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:15.953383 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1419/2830] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:15 2022] DELETE /identity/v3/projects/01a98b3c25264b86ae1e016303015650 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:15.954255 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:15.955642 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.975911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] MFA Rules not processed for user `312fba0e9b894db290fb2d48d0d704e9`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:15.977200 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:15.978849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:16.038026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6af646-8784-41b7-ab49-0bf6b252711b admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.052503 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1412/2831] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:16.064678 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.066238 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.073143 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:16.110951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.111926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.112202 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.112508 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] PATH_INFO: `/v3/users/010105791cdb4110bc848141920140f8` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.113786 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=010105791cdb4110bc848141920140f8)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.116915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.140381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a61615-e960-4054-9bb1-649dd4def536 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.144487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.145307 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.145594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.145880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.156177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.157653 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.166451 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010105791cdb4110bc848141920140f8'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.166792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1413/2832] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:16.167618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.169911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.181179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010105791cdb4110bc848141920140f8'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.186549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf23432-ac33-4b2a-a977-5332ea4da857 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.190949 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010105791cdb4110bc848141920140f8'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.191767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.192729 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.193106 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.193453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] PATH_INFO: `/v3/users/956a97369567406da4d743fd091938c8` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.194767 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=956a97369567406da4d743fd091938c8)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.196815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.200415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010105791cdb4110bc848141920140f8'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.201249 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-e50fd856-db8f-4bee-a9b9-39272fa8d511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010105791cdb4110bc848141920140f8'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.209050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1420/2833] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:15 2022] DELETE /identity/v3/users/010105791cdb4110bc848141920140f8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:16.254667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956a97369567406da4d743fd091938c8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.270080 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956a97369567406da4d743fd091938c8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.279129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956a97369567406da4d743fd091938c8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.289870 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956a97369567406da4d743fd091938c8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.291041 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956a97369567406da4d743fd091938c8'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.299279 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1414/2834] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:16 2022] DELETE /identity/v3/users/956a97369567406da4d743fd091938c8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:16.304084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.304381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.304668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.326458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:16.327852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.378960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:16.415762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1421/2835] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:16.419932 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.421741 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.426415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.428175 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.450768 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75462b0-f66c-415a-94b4-1d8e317d9c39 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.456456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.457372 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.457651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.458061 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] PATH_INFO: `/v3/projects/5878e03801294ffd825603b4ae70d2e2` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.459281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5878e03801294ffd825603b4ae70d2e2)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.461568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.513440 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab401-d05c-49cb-b513-bd5a33177aea None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.545529 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5878e03801294ffd825603b4ae70d2e2', 'request_id': 'req-886467e7-d640-49c3-a5a6-08de1558a7d4'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.550833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5878e03801294ffd825603b4ae70d2e2', 'request_id': 'req-886467e7-d640-49c3-a5a6-08de1558a7d4'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.558875 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5878e03801294ffd825603b4ae70d2e2', 'request_id': 'req-886467e7-d640-49c3-a5a6-08de1558a7d4'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.575053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cd473-d175-49ea-9872-c53d1d2ac829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.582851 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd473-d175-49ea-9872-c53d1d2ac829 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.583002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] The token cache is being invalidate because project 5878e03801294ffd825603b4ae70d2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:16.583091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd473-d175-49ea-9872-c53d1d2ac829 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.583197 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.583279 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd473-d175-49ea-9872-c53d1d2ac829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.583857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-886467e7-d640-49c3-a5a6-08de1558a7d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:16.585961 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1415/2836] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:16 2022] DELETE /identity/v3/projects/5878e03801294ffd825603b4ae70d2e2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:16.591417 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3324f-5bae-418d-9f3e-1a2c0e0b9d08 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.591714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3324f-5bae-418d-9f3e-1a2c0e0b9d08 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.591994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3324f-5bae-418d-9f3e-1a2c0e0b9d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.598455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:16.616015 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-19e3324f-5bae-418d-9f3e-1a2c0e0b9d08 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:16.617221 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-19e3324f-5bae-418d-9f3e-1a2c0e0b9d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.647699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cd473-d175-49ea-9872-c53d1d2ac829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.649248 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cd473-d175-49ea-9872-c53d1d2ac829 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.658964 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1422/2837] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:16.681776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1416/2838] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:16.690150 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d355188-b2cf-416a-9ee3-9699bc3ec21f None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.690444 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d355188-b2cf-416a-9ee3-9699bc3ec21f None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.690725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d355188-b2cf-416a-9ee3-9699bc3ec21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.712484 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-7d355188-b2cf-416a-9ee3-9699bc3ec21f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:16.714083 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-7d355188-b2cf-416a-9ee3-9699bc3ec21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.723144 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1423/2839] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:16.727825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3324f-5bae-418d-9f3e-1a2c0e0b9d08 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.738537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3324f-5bae-418d-9f3e-1a2c0e0b9d08 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.744110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.744900 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.745246 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.745589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.746537 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:16.747082 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.748810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.763996 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1417/2840] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:16.768053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d355188-b2cf-416a-9ee3-9699bc3ec21f None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.779630 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d355188-b2cf-416a-9ee3-9699bc3ec21f None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.784670 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.785379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.786138 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.786311 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.786937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.789194 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.824366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1424/2841] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:16 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:16.828412 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.838646 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56397f26-3ae5-47ae-ba8e-986954dff21c admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.903341 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.904134 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.904410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.904686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.905397 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.907665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.928154 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1418/2842] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:16 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:16.932333 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.943733 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1728e29-b723-4b05-aafe-a8cc2b726c85 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.949207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.949960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.950244 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.950569 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.951265 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.952915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.961297 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.963155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:16.970050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1425/2843] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:16.973901 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:16.978680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a6dec-7de9-48d7-8d2f-65bd850fdbf1 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.987409 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ad62b-48f8-41c4-8040-8a7a01afa980 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:16.989890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.991002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.991360 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.991839 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] PATH_INFO: `/v3/projects/0cc74f7624cd49658f030d963d8f5d75` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.992530 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:16.993196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc74f7624cd49658f030d963d8f5d75)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.993294 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:16.993483 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:16.993763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:16.994501 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:16.995177 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:16.996127 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.001862 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:17.008003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1426/2844] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:17.012373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.022170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9364f-57b9-427d-9acf-00eca16587d8 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.028866 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942c553-c40b-4735-bbab-e72b4e9664f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.029802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c553-c40b-4735-bbab-e72b4e9664f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.030215 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c553-c40b-4735-bbab-e72b4e9664f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.030499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c553-c40b-4735-bbab-e72b4e9664f0 admin admin] PATH_INFO: `/v3/projects/c9b7a333f98144b99c77d82c67097da3/users/97cba089bfbd49a39d965a24e9a32c52/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.046548 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc74f7624cd49658f030d963d8f5d75', 'request_id': 'req-8ed854c0-ed1d-4035-af1f-67c209427e73'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.046763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942c553-c40b-4735-bbab-e72b4e9664f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b7a333f98144b99c77d82c67097da3, user_id=97cba089bfbd49a39d965a24e9a32c52, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.049315 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942c553-c40b-4735-bbab-e72b4e9664f0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.053815 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc74f7624cd49658f030d963d8f5d75', 'request_id': 'req-8ed854c0-ed1d-4035-af1f-67c209427e73'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.060206 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc74f7624cd49658f030d963d8f5d75', 'request_id': 'req-8ed854c0-ed1d-4035-af1f-67c209427e73'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.063476 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1427/2845] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:17 2022] PUT /identity/v3/projects/c9b7a333f98144b99c77d82c67097da3/users/97cba089bfbd49a39d965a24e9a32c52/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:17.075122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] The token cache is being invalidate because project 0cc74f7624cd49658f030d963d8f5d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:17.075335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.075443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.075523 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.075769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.076084 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.077670 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1419/2846] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:16 2022] DELETE /identity/v3/projects/0cc74f7624cd49658f030d963d8f5d75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:17.081675 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.082980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.088787 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:17.103871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] MFA Rules not processed for user `97cba089bfbd49a39d965a24e9a32c52`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:17.105084 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.158110 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed854c0-ed1d-4035-af1f-67c209427e73 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.186006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1428/2847] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:17.197784 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.199926 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.217283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.218187 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.218843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.218843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] PATH_INFO: `/v3/users/5a1cf324c0614c5b9d6172ffa631bacf` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.219973 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1cf324c0614c5b9d6172ffa631bacf)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.221924 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.258813 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b2e10-c2fc-4adb-8bd4-d214ae77b08a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.262589 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.263293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.263638 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.263917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.270054 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1cf324c0614c5b9d6172ffa631bacf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.272811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.274253 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.282904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1cf324c0614c5b9d6172ffa631bacf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.283272 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1429/2848] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:17.289997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1cf324c0614c5b9d6172ffa631bacf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.303239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.304774 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.305525 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1cf324c0614c5b9d6172ffa631bacf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.306321 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-b4b85015-b73a-4d73-940e-edb97677473e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1cf324c0614c5b9d6172ffa631bacf'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.313142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1420/2849] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:17 2022] DELETE /identity/v3/users/5a1cf324c0614c5b9d6172ffa631bacf => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:17.322336 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc727a-ee0a-4227-8a1f-945a4786a42e service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.382184 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.383082 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.383380 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.383674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] PATH_INFO: `/v3/users/97cba089bfbd49a39d965a24e9a32c52` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.384959 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97cba089bfbd49a39d965a24e9a32c52)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.386924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.398088 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:17.437912 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97cba089bfbd49a39d965a24e9a32c52'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.451230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97cba089bfbd49a39d965a24e9a32c52'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.459843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97cba089bfbd49a39d965a24e9a32c52'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.469442 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97cba089bfbd49a39d965a24e9a32c52'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.470197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97cba089bfbd49a39d965a24e9a32c52'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.476885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1430/2850] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:17 2022] DELETE /identity/v3/users/97cba089bfbd49a39d965a24e9a32c52 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:17.481526 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.482255 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.482605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.500579 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:17.501906 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.527411 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:17.566774 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1421/2851] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:17.581765 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.583477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.612853 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.614431 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.631494 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d96c42-f321-4209-b4d5-1c419339fbba None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.637789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.639357 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.639680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.639959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] PATH_INFO: `/v3/projects/e140b54b55af4f3a94c50f071081a110` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.641275 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] RBAC: Authorizing `identity:delete_project(project_id=e140b54b55af4f3a94c50f071081a110)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.644283 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.658696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b462cc-699a-4040-95de-1847e267db67 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.662686 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edc491-cd3c-4e43-b6ae-aa9536fc2f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.663690 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edc491-cd3c-4e43-b6ae-aa9536fc2f24 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.664031 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edc491-cd3c-4e43-b6ae-aa9536fc2f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.664426 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edc491-cd3c-4e43-b6ae-aa9536fc2f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.676578 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edc491-cd3c-4e43-b6ae-aa9536fc2f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.678375 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edc491-cd3c-4e43-b6ae-aa9536fc2f24 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.690305 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1431/2852] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:17.706349 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e140b54b55af4f3a94c50f071081a110', 'request_id': 'req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.715674 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e140b54b55af4f3a94c50f071081a110', 'request_id': 'req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.724225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e140b54b55af4f3a94c50f071081a110', 'request_id': 'req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.743509 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] The token cache is being invalidate because project e140b54b55af4f3a94c50f071081a110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:17.743807 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.744717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-0832da25-76c1-4a8c-bd2f-5b849bb9dfde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:17.746455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1422/2853] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:17 2022] DELETE /identity/v3/projects/e140b54b55af4f3a94c50f071081a110 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:17.751510 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06a715-0181-41a5-9c23-b7099ebc5460 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.752182 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06a715-0181-41a5-9c23-b7099ebc5460 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.752535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06a715-0181-41a5-9c23-b7099ebc5460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.780389 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-4e06a715-0181-41a5-9c23-b7099ebc5460 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:17.781605 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4e06a715-0181-41a5-9c23-b7099ebc5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.880225 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4396a-d61c-4d64-b1ae-1ee1cf78e407 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.880514 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4396a-d61c-4d64-b1ae-1ee1cf78e407 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.880792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4396a-d61c-4d64-b1ae-1ee1cf78e407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.882486 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1432/2854] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:17.902115 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-67f4396a-d61c-4d64-b1ae-1ee1cf78e407 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:17.903704 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-67f4396a-d61c-4d64-b1ae-1ee1cf78e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:17.916618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1423/2855] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:17.918105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06a715-0181-41a5-9c23-b7099ebc5460 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.929173 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06a715-0181-41a5-9c23-b7099ebc5460 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.935318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.936043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.936326 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.936601 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.937844 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:17.938165 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.939985 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.953462 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1433/2856] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:17.958119 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4396a-d61c-4d64-b1ae-1ee1cf78e407 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.974731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4396a-d61c-4d64-b1ae-1ee1cf78e407 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:17.980029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:17.980938 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:17.981289 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:17.985077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:17.985956 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:17.989190 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:17.991213 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:17.998430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:18.026738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1424/2857] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:17 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:18.027594 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e8423-f52b-4688-a95b-f1bdfcaacc8e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.030659 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.048171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:18.116746 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.116965 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71b217-9e40-4cd2-91d9-3f9a46eb7b15 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.117050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.117119 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.117623 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] PATH_INFO: `/v3/projects/c9b7a333f98144b99c77d82c67097da3` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.121106 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b7a333f98144b99c77d82c67097da3)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.122381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36704769-8175-414c-9262-36a3548a437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.123242 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36704769-8175-414c-9262-36a3548a437f admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.123568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36704769-8175-414c-9262-36a3548a437f admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.123880 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36704769-8175-414c-9262-36a3548a437f admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.124460 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.124594 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36704769-8175-414c-9262-36a3548a437f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.127107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36704769-8175-414c-9262-36a3548a437f admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.156867 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1425/2858] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:18.157901 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36704769-8175-414c-9262-36a3548a437f admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.170094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36704769-8175-414c-9262-36a3548a437f admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.175120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.175872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.176242 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.176683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.177763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.179423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.190781 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1426/2859] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:18.193029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b7a333f98144b99c77d82c67097da3', 'request_id': 'req-64bba753-b209-4044-bc4d-7cf1f39d4ebb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:18.196477 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.201015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b7a333f98144b99c77d82c67097da3', 'request_id': 'req-64bba753-b209-4044-bc4d-7cf1f39d4ebb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:18.202424 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:18.209155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ee685-8685-4976-bc33-374b70d7d547 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.209331 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b7a333f98144b99c77d82c67097da3', 'request_id': 'req-64bba753-b209-4044-bc4d-7cf1f39d4ebb'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:18.228350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] The token cache is being invalidate because project c9b7a333f98144b99c77d82c67097da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:18.228864 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:18.229550 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-64bba753-b209-4044-bc4d-7cf1f39d4ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:18.231407 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1434/2860] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:17 2022] DELETE /identity/v3/projects/c9b7a333f98144b99c77d82c67097da3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:18.237377 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20847ef-3914-4b93-a842-a9e2c58d7109 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.237587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20847ef-3914-4b93-a842-a9e2c58d7109 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.238051 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20847ef-3914-4b93-a842-a9e2c58d7109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.258980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-f20847ef-3914-4b93-a842-a9e2c58d7109 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:18.260262 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f20847ef-3914-4b93-a842-a9e2c58d7109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:18.275663 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.276347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.276632 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.276926 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.277751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.279423 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.289582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1427/2861] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:18.293911 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.295732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:18.309348 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa10212-002b-474a-8198-235d2914ecd0 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.314368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecd5a7-14b1-443a-a3c5-12637d176d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.315271 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecd5a7-14b1-443a-a3c5-12637d176d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.315590 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecd5a7-14b1-443a-a3c5-12637d176d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.315873 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecd5a7-14b1-443a-a3c5-12637d176d89 admin admin] PATH_INFO: `/v3/projects/c6b197193c1047408e6966601b6e74b6/users/ecb60292227945a4a3c0165f29458437/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.323734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1435/2862] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:18.330966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecd5a7-14b1-443a-a3c5-12637d176d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b197193c1047408e6966601b6e74b6, user_id=ecb60292227945a4a3c0165f29458437, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.333724 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecd5a7-14b1-443a-a3c5-12637d176d89 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.349171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1428/2863] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:18 2022] PUT /identity/v3/projects/c6b197193c1047408e6966601b6e74b6/users/ecb60292227945a4a3c0165f29458437/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:18.390470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102a5cc-f064-4cd1-8964-ea98ded1e2d9 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.390923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102a5cc-f064-4cd1-8964-ea98ded1e2d9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.391404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102a5cc-f064-4cd1-8964-ea98ded1e2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.411270 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-e102a5cc-f064-4cd1-8964-ea98ded1e2d9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:18.412571 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e102a5cc-f064-4cd1-8964-ea98ded1e2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:18.413909 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1d3e4-ae46-4b68-ad56-e916414ac47c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.414205 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1d3e4-ae46-4b68-ad56-e916414ac47c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.414490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1d3e4-ae46-4b68-ad56-e916414ac47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.453904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-9bd1d3e4-ae46-4b68-ad56-e916414ac47c None None] MFA Rules not processed for user `ecb60292227945a4a3c0165f29458437`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:18.455072 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9bd1d3e4-ae46-4b68-ad56-e916414ac47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:18.459768 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:18.479613 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1436/2864] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:18.484159 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102a5cc-f064-4cd1-8964-ea98ded1e2d9 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.493431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102a5cc-f064-4cd1-8964-ea98ded1e2d9 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.498655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.499335 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.499646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.500275 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] PATH_INFO: `/v3/domains` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.500709 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97009) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:18.501019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.502668 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.515324 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1437/2865] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 13:29:18.519705 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.530787 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e462a7a-b321-4c23-8257-1dfa18eaf6bd admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.532907 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1429/2866] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:18.535700 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.536376 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.536661 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.536940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.537263 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1d3e4-ae46-4b68-ad56-e916414ac47c None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.537640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.539614 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.548247 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1d3e4-ae46-4b68-ad56-e916414ac47c None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.553913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.554555 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.554841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.555122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] PATH_INFO: `/v3/projects` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.555851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.557782 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.567477 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1438/2867] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:18.571291 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.590199 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1430/2868] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:18.591444 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.637197 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24567-8108-41ca-8054-aec4fb950e9d admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.639155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:18.658582 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cb06b-6bc9-4bd5-9c9f-20745c558636 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.663443 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.664105 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.664392 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.664669 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.665378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.667353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.689342 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1431/2869] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:18.693484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.696395 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.697131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.697474 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.697758 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] PATH_INFO: `/v3/users` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.698506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.700801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.704816 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53080ca5-ca50-485e-be6b-3a75bfadcc02 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.710446 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.711135 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.711455 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.711737 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.712493 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.714175 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.726220 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1439/2870] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:18.726414 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1432/2871] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:18.727500 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.729655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.739045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceca08a-e6f5-4602-8b9f-5dc3848c08e9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.740738 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863860-8b62-4db3-9382-5cd99e7f622e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.743745 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.744490 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.744722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.745019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.745751 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.745854 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.746803 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.747089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.747395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] PATH_INFO: `/v3/projects/f237b1dee5844a9d97b3a6bc441f42b4/users/4f229403022c44c39309f16066f419f0/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.748393 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.762198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1440/2872] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:18.763911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.766796 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f237b1dee5844a9d97b3a6bc441f42b4, user_id=4f229403022c44c39309f16066f419f0, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.769500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.773728 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6f43-a3e5-49bf-a535-04fe4ce55180 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.779207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.779928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.780198 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.780620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.781356 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.782952 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.794540 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1441/2873] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:18.796303 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1433/2874] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:18 2022] PUT /identity/v3/projects/f237b1dee5844a9d97b3a6bc441f42b4/users/4f229403022c44c39309f16066f419f0/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:18.799105 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.800145 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.810114 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6718ca7-6eef-48da-8aca-14d92ee07dc9 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.839541 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:18.867990 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37707112-703f-499b-a142-dc734d515dcf admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.873745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.873885 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c76c7b-9166-4843-bf43-3a26ae444a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.874425 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.874713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.874827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c76c7b-9166-4843-bf43-3a26ae444a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.874992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.875076 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c76c7b-9166-4843-bf43-3a26ae444a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.875325 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c76c7b-9166-4843-bf43-3a26ae444a35 admin admin] PATH_INFO: `/v3/projects/d379d8a3623f4f899ba398ca64105186/users/5dfaac7738854799b366623cac5d5138/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.875761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.877384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.888338 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1434/2875] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:18.892572 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:18.894079 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c76c7b-9166-4843-bf43-3a26ae444a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d379d8a3623f4f899ba398ca64105186, user_id=5dfaac7738854799b366623cac5d5138, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.896801 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c76c7b-9166-4843-bf43-3a26ae444a35 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.902304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c133f0-c7ab-462a-b13b-23a21a847ce9 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:18.907827 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132dd6e-9810-4b34-92fe-b6b7ecc2b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:18.908732 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132dd6e-9810-4b34-92fe-b6b7ecc2b266 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.909094 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132dd6e-9810-4b34-92fe-b6b7ecc2b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.909368 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132dd6e-9810-4b34-92fe-b6b7ecc2b266 admin admin] PATH_INFO: `/v3/projects/f237b1dee5844a9d97b3a6bc441f42b4/users/4f229403022c44c39309f16066f419f0/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.911331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132dd6e-9810-4b34-92fe-b6b7ecc2b266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f237b1dee5844a9d97b3a6bc441f42b4, user_id=4f229403022c44c39309f16066f419f0, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:18.911463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1442/2876] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:18 2022] PUT /identity/v3/projects/d379d8a3623f4f899ba398ca64105186/users/5dfaac7738854799b366623cac5d5138/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:18.913997 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132dd6e-9810-4b34-92fe-b6b7ecc2b266 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:18.921524 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.921816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.922097 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.927986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1435/2877] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:18 2022] PUT /identity/v3/projects/f237b1dee5844a9d97b3a6bc441f42b4/users/4f229403022c44c39309f16066f419f0/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:18.939022 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:18.939302 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:18.939605 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:18.950790 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] MFA Rules not processed for user `5dfaac7738854799b366623cac5d5138`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:18.952047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:18.967621 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] MFA Rules not processed for user `4f229403022c44c39309f16066f419f0`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:18.968752 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:19.018953 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1443/2878] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:19.031886 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:19.033286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:19.039445 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1436/2879] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:19.052095 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:19.053491 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:19.059998 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:19.094012 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ee072-02e1-440a-abfe-8880627d43ce None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:19.097719 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:19.098381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:19.098669 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:19.098945 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:19.108027 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:19.109478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:19.112623 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65817d-41df-4f50-be6c-002030efb416 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:19.122254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:19.122913 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:19.123479 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:19.123754 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:19.124568 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1444/2880] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:19.134270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:19.135758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:19.146013 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1437/2881] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:19.300209 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:19.310199 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc03b-f45b-4dcb-83e5-3550a7198391 service cinder] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:19.313902 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181729d-e70f-4599-ad2c-052f0e02a97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:19.314553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181729d-e70f-4599-ad2c-052f0e02a97c service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:19.314840 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181729d-e70f-4599-ad2c-052f0e02a97c service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:19.315111 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181729d-e70f-4599-ad2c-052f0e02a97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:19.362974 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:19.380871 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181729d-e70f-4599-ad2c-052f0e02a97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:19.382514 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181729d-e70f-4599-ad2c-052f0e02a97c service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:19.393026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1445/2882] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:24.251353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:24.342029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fea61-dcaf-450e-a202-21ae21dbe1e9 service cinder] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:24.349184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:24.350695 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:24.351152 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:24.351640 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] PATH_INFO: `/v3/users/ecb60292227945a4a3c0165f29458437` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:24.353688 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb60292227945a4a3c0165f29458437)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:24.357613 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:24.420682 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb60292227945a4a3c0165f29458437'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:24.436244 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb60292227945a4a3c0165f29458437'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:24.446030 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb60292227945a4a3c0165f29458437'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:24.455686 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb60292227945a4a3c0165f29458437'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:24.456589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb60292227945a4a3c0165f29458437'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:24.464887 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1438/2883] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:24 2022] DELETE /identity/v3/users/ecb60292227945a4a3c0165f29458437 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:24.469792 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:24.470299 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:24.470402 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:24.489039 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:24.490282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:24.555845 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1446/2884] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:24.565902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:24.567442 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:24.603319 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:24.634476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8187fc-3694-4197-9435-3db1b484cb0a admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:24.639121 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:24.639910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:24.640217 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:24.640495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:24.651717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:24.653254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:24.663792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1439/2885] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:24.957054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:24.958536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:24.971240 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45433457-20d2-4c2a-8677-e5388770dff7 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:24.976518 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:24.977499 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:24.977856 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:24.978205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] PATH_INFO: `/v3/projects/c6b197193c1047408e6966601b6e74b6` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:24.979718 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b197193c1047408e6966601b6e74b6)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:24.981975 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:25.033928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b197193c1047408e6966601b6e74b6', 'request_id': 'req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.040521 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b197193c1047408e6966601b6e74b6', 'request_id': 'req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.048609 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b197193c1047408e6966601b6e74b6', 'request_id': 'req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.063888 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] The token cache is being invalidate because project c6b197193c1047408e6966601b6e74b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:25.064300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.064971 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.066608 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1447/2886] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:24 2022] DELETE /identity/v3/projects/c6b197193c1047408e6966601b6e74b6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:25.070696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:25.080910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da2790-ad6d-4842-b400-5babe65f5315 service neutron] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:25.150422 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:25.151234 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:25.151549 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:25.151825 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] PATH_INFO: `/v3/users/4f229403022c44c39309f16066f419f0` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:25.153322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f229403022c44c39309f16066f419f0)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:25.155325 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:25.208959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f229403022c44c39309f16066f419f0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.223011 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f229403022c44c39309f16066f419f0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.230593 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f229403022c44c39309f16066f419f0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.240750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f229403022c44c39309f16066f419f0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.241681 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-ebdb74db-3dc7-4b8d-862f-1e2f8c890247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f229403022c44c39309f16066f419f0'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.249994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1440/2887] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:25 2022] DELETE /identity/v3/users/4f229403022c44c39309f16066f419f0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:25.544586 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:25.546341 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:25.560251 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf0c5e-34bc-435e-bf77-c5223dc7fb18 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:25.630133 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:25.631057 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:25.631338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:25.631644 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] PATH_INFO: `/v3/projects/f237b1dee5844a9d97b3a6bc441f42b4` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:25.632956 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f237b1dee5844a9d97b3a6bc441f42b4)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:25.634869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:25.644212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:25.698591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f237b1dee5844a9d97b3a6bc441f42b4', 'request_id': 'req-2b0cc707-56b7-4548-8cf1-2c9e45e96324'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.707227 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f237b1dee5844a9d97b3a6bc441f42b4', 'request_id': 'req-2b0cc707-56b7-4548-8cf1-2c9e45e96324'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.714158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f237b1dee5844a9d97b3a6bc441f42b4', 'request_id': 'req-2b0cc707-56b7-4548-8cf1-2c9e45e96324'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.729793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] The token cache is being invalidate because project f237b1dee5844a9d97b3a6bc441f42b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:25.730090 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.730858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-2b0cc707-56b7-4548-8cf1-2c9e45e96324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:25.733459 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1448/2888] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:25 2022] DELETE /identity/v3/projects/f237b1dee5844a9d97b3a6bc441f42b4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:25.739328 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f987b-53c0-451e-b36c-e19e64889078 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:25.739851 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f987b-53c0-451e-b36c-e19e64889078 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:25.739981 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f987b-53c0-451e-b36c-e19e64889078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:25.759301 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-837f987b-53c0-451e-b36c-e19e64889078 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:25.760578 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-837f987b-53c0-451e-b36c-e19e64889078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:25.815835 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:25.834315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1441/2889] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:25.844008 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9391c55-e970-47f5-babd-b4bf516c1cfb None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:25.844295 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9391c55-e970-47f5-babd-b4bf516c1cfb None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:25.844585 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9391c55-e970-47f5-babd-b4bf516c1cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:25.864882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c9391c55-e970-47f5-babd-b4bf516c1cfb None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:25.866102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c9391c55-e970-47f5-babd-b4bf516c1cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:25.874852 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1449/2890] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:25.879180 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f987b-53c0-451e-b36c-e19e64889078 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:25.889747 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f987b-53c0-451e-b36c-e19e64889078 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:25.894959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:25.895673 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:25.895957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:25.896232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:25.897005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:25.897314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:25.898972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:25.914281 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1442/2891] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:25.918987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9391c55-e970-47f5-babd-b4bf516c1cfb None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:25.929205 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9391c55-e970-47f5-babd-b4bf516c1cfb None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:25.934620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:25.935713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:25.936004 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:25.936290 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:25.937003 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:25.938929 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:25.975481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1450/2892] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:25 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:25.976648 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:25.988314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daee88-15cc-4249-93a1-09e3f98fca11 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:26.056971 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:26.057662 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:26.057947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:26.058227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:26.059628 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:26.061655 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:26.084044 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1443/2893] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:25 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:26.088203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:26.099286 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48d56-233a-4b22-a108-bfaa62c7ab57 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:26.103880 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:26.104553 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:26.104858 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:26.105103 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:26.106923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:26.108565 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:26.120916 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1451/2894] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:26.125173 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:26.135387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd95cb-8d0b-48e5-b637-789be031df85 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:26.140992 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2853a-6b76-4c09-be3a-5abe2e7a6a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:26.141672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2853a-6b76-4c09-be3a-5abe2e7a6a57 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:26.141955 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2853a-6b76-4c09-be3a-5abe2e7a6a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:26.142232 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2853a-6b76-4c09-be3a-5abe2e7a6a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:26.142972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2853a-6b76-4c09-be3a-5abe2e7a6a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:26.144680 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2853a-6b76-4c09-be3a-5abe2e7a6a57 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:26.155508 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1444/2895] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:26.160277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:26.171672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe0444-e4e9-400b-b7da-e6281c56c911 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:26.177827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:26.178849 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:26.179137 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:26.179456 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] PATH_INFO: `/v3/projects/43a0cbd9ac674da291208d8cc3f595b6/users/a30ffc63d64347889b597416440eedc7/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:26.196723 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a0cbd9ac674da291208d8cc3f595b6, user_id=a30ffc63d64347889b597416440eedc7, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:26.200479 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:26.214610 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1452/2896] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:26 2022] PUT /identity/v3/projects/43a0cbd9ac674da291208d8cc3f595b6/users/a30ffc63d64347889b597416440eedc7/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:26.225950 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:26.226243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:26.226546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:26.256776 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] MFA Rules not processed for user `a30ffc63d64347889b597416440eedc7`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:26.258016 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:26.325693 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1445/2897] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:26.337212 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:26.339025 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:26.411281 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564abd7-2194-4a02-af1e-b448c994316e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:26.415362 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05da57-53cc-44b1-8603-040d60634e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:26.416063 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05da57-53cc-44b1-8603-040d60634e43 service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:26.416348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05da57-53cc-44b1-8603-040d60634e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:26.416622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05da57-53cc-44b1-8603-040d60634e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:26.426026 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05da57-53cc-44b1-8603-040d60634e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:26.427398 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05da57-53cc-44b1-8603-040d60634e43 service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:26.436720 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1453/2898] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:30.414792 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:30.416702 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:30.473454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:30.495631 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcce75b-903a-4a17-884c-6138a156c12b None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:30.500890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:30.501810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:30.502052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:30.502347 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] PATH_INFO: `/v3/users/5dfaac7738854799b366623cac5d5138` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:30.503604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dfaac7738854799b366623cac5d5138)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:30.505793 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:30.556722 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dfaac7738854799b366623cac5d5138'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:30.572451 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dfaac7738854799b366623cac5d5138'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:30.581077 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dfaac7738854799b366623cac5d5138'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:30.589032 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dfaac7738854799b366623cac5d5138'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:30.589980 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dfaac7738854799b366623cac5d5138'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:30.598309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1446/2899] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:30 2022] DELETE /identity/v3/users/5dfaac7738854799b366623cac5d5138 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:30.604183 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:30.604678 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:30.605024 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:30.624646 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:30.625843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:30.665081 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:30.705607 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1454/2900] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:30.712859 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:30.716510 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:30.776568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba79a-1af7-4fa4-968a-fd2b80196ba5 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:30.780889 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab39f5-cba6-440f-a52b-6c84d3ee8d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:30.781193 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab39f5-cba6-440f-a52b-6c84d3ee8d2a service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:30.781480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab39f5-cba6-440f-a52b-6c84d3ee8d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:30.781761 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab39f5-cba6-440f-a52b-6c84d3ee8d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:30.791436 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab39f5-cba6-440f-a52b-6c84d3ee8d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:30.792915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab39f5-cba6-440f-a52b-6c84d3ee8d2a service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:30.803487 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1447/2901] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:31.078536 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.080102 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:31.096047 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14481d7-42ac-4880-a278-5b58b17e248a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.101520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.102415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.102711 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.102996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] PATH_INFO: `/v3/projects/d379d8a3623f4f899ba398ca64105186` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.104439 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=d379d8a3623f4f899ba398ca64105186)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.107153 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.165316 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd379d8a3623f4f899ba398ca64105186', 'request_id': 'req-9daa9f13-adc5-4231-a33a-acec8b1d7dab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:31.173344 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd379d8a3623f4f899ba398ca64105186', 'request_id': 'req-9daa9f13-adc5-4231-a33a-acec8b1d7dab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:31.181350 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd379d8a3623f4f899ba398ca64105186', 'request_id': 'req-9daa9f13-adc5-4231-a33a-acec8b1d7dab'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:31.196622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] The token cache is being invalidate because project d379d8a3623f4f899ba398ca64105186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:31.196914 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:31.197547 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-9daa9f13-adc5-4231-a33a-acec8b1d7dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:31.199473 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1455/2902] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:31 2022] DELETE /identity/v3/projects/d379d8a3623f4f899ba398ca64105186 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:31.204759 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b613d-8b26-46ab-98e0-e8b79429d9a8 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.205138 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b613d-8b26-46ab-98e0-e8b79429d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.205463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b613d-8b26-46ab-98e0-e8b79429d9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.224282 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-050b613d-8b26-46ab-98e0-e8b79429d9a8 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:31.225501 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-050b613d-8b26-46ab-98e0-e8b79429d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:31.287256 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1448/2903] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:31.296724 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c6ad6-cbf8-4f74-be29-ca4017c22b01 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.297011 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c6ad6-cbf8-4f74-be29-ca4017c22b01 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.297283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c6ad6-cbf8-4f74-be29-ca4017c22b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.317239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-922c6ad6-cbf8-4f74-be29-ca4017c22b01 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:31.318465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-922c6ad6-cbf8-4f74-be29-ca4017c22b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:31.328427 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1456/2904] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:31.332933 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b613d-8b26-46ab-98e0-e8b79429d9a8 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.343416 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b613d-8b26-46ab-98e0-e8b79429d9a8 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.349261 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.349936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.350219 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.350492 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.351227 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:31.351557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.353211 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.354753 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:31.369161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1449/2905] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:31.373820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c6ad6-cbf8-4f74-be29-ca4017c22b01 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.383943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c6ad6-cbf8-4f74-be29-ca4017c22b01 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.390011 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.390680 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.390968 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.391239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.391988 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.393917 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.427403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1457/2906] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:31 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:31.428717 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.440109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57907455-fa62-48a7-b3a3-9af5b3caf40e admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.508832 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.509495 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.509771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.510052 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.510763 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.513743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.536223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1450/2907] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:31 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:31.540372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.552860 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138c8d4-e6c4-499f-becf-bc52fccf0d9e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.557572 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.558318 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.558779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.559752 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.560537 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.562179 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.574080 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1458/2908] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:31.580111 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.590390 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39164b9-2f71-41ca-a26d-2a035314a6c8 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.596497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72768c5c-969e-4ef7-bba3-3813c43f384b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.597331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72768c5c-969e-4ef7-bba3-3813c43f384b admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.597616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72768c5c-969e-4ef7-bba3-3813c43f384b admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.597890 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72768c5c-969e-4ef7-bba3-3813c43f384b admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.598677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72768c5c-969e-4ef7-bba3-3813c43f384b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.600449 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72768c5c-969e-4ef7-bba3-3813c43f384b admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.612684 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1451/2909] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:31.616987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.629372 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73235b89-6cbb-452a-8dea-ab8b20c487c0 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.634883 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.635924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.636273 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.636620 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] PATH_INFO: `/v3/projects/75b686945ee4451daae3d981f261061f/users/a84f39ff866e48038c9d36188824c9e5/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.660277 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b686945ee4451daae3d981f261061f, user_id=a84f39ff866e48038c9d36188824c9e5, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.663043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.681688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1459/2910] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:31 2022] PUT /identity/v3/projects/75b686945ee4451daae3d981f261061f/users/a84f39ff866e48038c9d36188824c9e5/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:31.693616 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.694091 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.694568 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.735700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] MFA Rules not processed for user `a84f39ff866e48038c9d36188824c9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:31.737051 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:31.825552 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1452/2911] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:31.858045 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:31.859867 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:31.925699 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d214-1795-4589-bfa6-ef31f074c464 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:31.929935 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854dff3-e8b4-4f05-a341-aa76f73e8b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1315c22b7db84afd9ef41bf853b9819b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:31.930667 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854dff3-e8b4-4f05-a341-aa76f73e8b7b service cinder] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:31.930965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854dff3-e8b4-4f05-a341-aa76f73e8b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:31.931230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854dff3-e8b4-4f05-a341-aa76f73e8b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:31.943385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854dff3-e8b4-4f05-a341-aa76f73e8b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:31.944996 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854dff3-e8b4-4f05-a341-aa76f73e8b7b service cinder] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:31.951923 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:31.965915 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1460/2912] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:29:32.893604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:32.895305 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:32.984139 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751503-98c0-4392-a791-5ab9309cf3b9 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:32.989604 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:32.990480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:32.990758 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:32.991029 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] PATH_INFO: `/v3/users/a30ffc63d64347889b597416440eedc7` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:32.992476 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30ffc63d64347889b597416440eedc7)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:32.994456 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:32.997210 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:33.050736 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30ffc63d64347889b597416440eedc7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.063396 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30ffc63d64347889b597416440eedc7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.072672 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30ffc63d64347889b597416440eedc7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.082769 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30ffc63d64347889b597416440eedc7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.083490 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30ffc63d64347889b597416440eedc7'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.090120 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1453/2913] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:32 2022] DELETE /identity/v3/users/a30ffc63d64347889b597416440eedc7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:33.095628 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:33.096147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:33.096233 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:33.118940 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:33.123672 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:33.189924 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1461/2914] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:33.199947 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:33.201598 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:33.267904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441ff85-9e04-450e-9a23-bd53129f3800 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:33.271118 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8103d-051d-4375-891a-c4cf0e0091b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:33.271836 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8103d-051d-4375-891a-c4cf0e0091b9 service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:33.272129 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8103d-051d-4375-891a-c4cf0e0091b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:33.272410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8103d-051d-4375-891a-c4cf0e0091b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:33.282668 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8103d-051d-4375-891a-c4cf0e0091b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:33.284168 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8103d-051d-4375-891a-c4cf0e0091b9 service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:33.294415 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1454/2915] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:33.616482 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:33.618121 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:33.636763 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b416c1-fa87-4483-9253-b502d2cad9a9 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:33.643382 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:33.644480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:33.644827 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:33.645145 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] PATH_INFO: `/v3/projects/43a0cbd9ac674da291208d8cc3f595b6` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:33.646533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a0cbd9ac674da291208d8cc3f595b6)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:33.648598 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:33.698015 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a0cbd9ac674da291208d8cc3f595b6', 'request_id': 'req-68d780bf-6d3b-44a1-a544-d27f9114ce01'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.707246 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a0cbd9ac674da291208d8cc3f595b6', 'request_id': 'req-68d780bf-6d3b-44a1-a544-d27f9114ce01'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.713365 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a0cbd9ac674da291208d8cc3f595b6', 'request_id': 'req-68d780bf-6d3b-44a1-a544-d27f9114ce01'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.726308 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] The token cache is being invalidate because project 43a0cbd9ac674da291208d8cc3f595b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:33.726599 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.727218 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-68d780bf-6d3b-44a1-a544-d27f9114ce01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:33.728830 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1462/2916] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:33 2022] DELETE /identity/v3/projects/43a0cbd9ac674da291208d8cc3f595b6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:33.734843 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4693664-5969-4e78-b44c-65d9b01c8c99 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:33.735144 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4693664-5969-4e78-b44c-65d9b01c8c99 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:33.735453 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4693664-5969-4e78-b44c-65d9b01c8c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:33.753447 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-d4693664-5969-4e78-b44c-65d9b01c8c99 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:33.754667 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d4693664-5969-4e78-b44c-65d9b01c8c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:33.826141 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1455/2917] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:33.835172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591c2e3-9ded-40f8-a1c7-6e973e5037c2 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:33.835493 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591c2e3-9ded-40f8-a1c7-6e973e5037c2 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:33.835771 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591c2e3-9ded-40f8-a1c7-6e973e5037c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:33.867053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-5591c2e3-9ded-40f8-a1c7-6e973e5037c2 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:33.868797 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-5591c2e3-9ded-40f8-a1c7-6e973e5037c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:33.879502 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1463/2918] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:33.884245 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4693664-5969-4e78-b44c-65d9b01c8c99 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:33.895534 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4693664-5969-4e78-b44c-65d9b01c8c99 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:33.901025 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:33.901805 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:33.902122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:33.902463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:33.903270 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=97010) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 13:29:33.903609 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:33.905418 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:33.919745 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1456/2919] 10.222.0.87 () {58 vars in 1152 bytes} [Fri Jun 17 13:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 13:29:33.924478 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591c2e3-9ded-40f8-a1c7-6e973e5037c2 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:33.934171 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591c2e3-9ded-40f8-a1c7-6e973e5037c2 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:33.939300 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:33.940019 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:33.940312 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:33.940637 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:33.941379 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:33.943392 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:33.981951 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1464/2920] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:33 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 13:29:33.986331 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:33.999512 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c0862-570b-4781-8fa6-4d175ef5f9e4 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:34.009181 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:34.068318 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:34.069020 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:34.069311 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:34.069617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:34.070322 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:34.072304 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:34.094683 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1457/2921] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:33 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:34.098663 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:34.109769 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8657b8-a635-44f2-b753-a48c17f3685c admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:34.115050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:34.115773 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:34.116062 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:34.116343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:34.117095 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:34.118708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:34.133391 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1465/2922] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:34.137458 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:34.148299 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f64e-d74a-48c6-831d-3ded0c45c6c2 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:34.153596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc3674-4e7e-439c-8e84-a1cb394988bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:34.154298 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3674-4e7e-439c-8e84-a1cb394988bc admin admin] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:34.154591 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3674-4e7e-439c-8e84-a1cb394988bc admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:34.154927 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3674-4e7e-439c-8e84-a1cb394988bc admin admin] PATH_INFO: `/v3/roles` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:34.155700 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc3674-4e7e-439c-8e84-a1cb394988bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:34.157370 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc3674-4e7e-439c-8e84-a1cb394988bc admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:34.168387 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1458/2923] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 13:29:34.172911 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:34.183636 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d66bb2-7cf0-4d59-98ce-a08b7a67bbe3 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:34.188542 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:34.189445 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:34.189730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:34.190006 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] PATH_INFO: `/v3/projects/362880716d884b098bab99cfcb5ed2aa/users/7e360d7c3b634c9eb26f1068b22ea56b/roles/fed756b9361747358b9a5948eb9a1442` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:34.206158 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=362880716d884b098bab99cfcb5ed2aa, user_id=7e360d7c3b634c9eb26f1068b22ea56b, role_id=fed756b9361747358b9a5948eb9a1442)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:34.208907 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:34.226412 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1466/2924] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:34 2022] PUT /identity/v3/projects/362880716d884b098bab99cfcb5ed2aa/users/7e360d7c3b634c9eb26f1068b22ea56b/roles/fed756b9361747358b9a5948eb9a1442 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:34.233618 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bbd8-4d01-4ae7-9e0a-c3e00aeb949c None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:34.233878 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bbd8-4d01-4ae7-9e0a-c3e00aeb949c None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:34.234128 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bbd8-4d01-4ae7-9e0a-c3e00aeb949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:34.253176 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-f2d2bbd8-4d01-4ae7-9e0a-c3e00aeb949c None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:34.254391 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-f2d2bbd8-4d01-4ae7-9e0a-c3e00aeb949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:34.322810 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1459/2925] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:34.333009 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:34.335043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:34.358054 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:34.400110 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5943b0-5445-4609-a1e7-40bc098866a7 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:34.404236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:34.404884 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:34.405172 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:34.405533 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:34.416091 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:34.417549 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:34.427877 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1467/2926] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:38.949009 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec168b-b348-430f-9431-95042e713c1d None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:38.949473 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec168b-b348-430f-9431-95042e713c1d None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:38.949557 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec168b-b348-430f-9431-95042e713c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:38.983394 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c2ec168b-b348-430f-9431-95042e713c1d None None] MFA Rules not processed for user `7e360d7c3b634c9eb26f1068b22ea56b`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:38.984902 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c2ec168b-b348-430f-9431-95042e713c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:39.065384 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1460/2927] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:39.070306 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:39.080713 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079709a8-7a72-4ab1-88e1-bdddc62fe63c service neutron] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:39.085480 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:39.086431 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:39.086718 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:39.086995 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:39.087749 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:39.089876 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:39.119987 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1468/2928] 10.222.0.87 () {60 vars in 1152 bytes} [Fri Jun 17 13:29:39 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:39.124828 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec168b-b348-430f-9431-95042e713c1d None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:39.136070 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec168b-b348-430f-9431-95042e713c1d None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:39.155596 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:39.204849 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:39.205558 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:39.205864 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:39.206140 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] PATH_INFO: `/v3/users` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:39.206852 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:39.208871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:39.234461 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1461/2929] 10.222.0.87 () {60 vars in 1143 bytes} [Fri Jun 17 13:29:39 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 13:29:39.235960 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:39.246980 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a20d-bbd6-44d8-bf1a-cf4ec78a4e8e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:39.252020 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:39.252820 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:39.253203 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:39.253622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:39.254535 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:39.256283 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:39.273287 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1469/2930] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:39.276538 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:39.289309 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282f31d-aa1d-4545-a7fc-fd5fec5f839d admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:39.294005 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:39.294920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:39.295516 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:39.295762 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] PATH_INFO: `/v3/projects/f390ded3fbe64c83b577ebcf11af278c/users/cd54bb2ffebc46f08f2b57cb60122376/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:39.312296 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f390ded3fbe64c83b577ebcf11af278c, user_id=cd54bb2ffebc46f08f2b57cb60122376, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:39.315547 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:39.332081 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1462/2931] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:39 2022] PUT /identity/v3/projects/f390ded3fbe64c83b577ebcf11af278c/users/cd54bb2ffebc46f08f2b57cb60122376/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 13:29:39.336289 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:39.347725 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470779e-4263-4165-97de-0843f68466f4 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:39.408722 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcfeef-cb06-4a9e-b1f5-8fba7e4fc2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:39.409411 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfeef-cb06-4a9e-b1f5-8fba7e4fc2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:39.409697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfeef-cb06-4a9e-b1f5-8fba7e4fc2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:39.409976 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcfeef-cb06-4a9e-b1f5-8fba7e4fc2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:39.410730 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcfeef-cb06-4a9e-b1f5-8fba7e4fc2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:39.412366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcfeef-cb06-4a9e-b1f5-8fba7e4fc2c3 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:39.422861 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1470/2932] 10.222.0.87 () {58 vars in 1121 bytes} [Fri Jun 17 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 13:29:39.427731 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:39.439243 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f54d7-bc5b-41ad-b7e9-80f13c981b95 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:39.444410 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:39.445315 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] REQUEST_METHOD: `PUT` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:39.445617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:39.445872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] PATH_INFO: `/v3/projects/f390ded3fbe64c83b577ebcf11af278c/users/cd54bb2ffebc46f08f2b57cb60122376/roles/4a6b9a3327544100929540645a797d97` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:39.447871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f390ded3fbe64c83b577ebcf11af278c, user_id=cd54bb2ffebc46f08f2b57cb60122376, role_id=4a6b9a3327544100929540645a797d97)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:39.450625 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:39.463696 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1463/2933] 10.222.0.87 () {60 vars in 1482 bytes} [Fri Jun 17 13:29:39 2022] PUT /identity/v3/projects/f390ded3fbe64c83b577ebcf11af278c/users/cd54bb2ffebc46f08f2b57cb60122376/roles/4a6b9a3327544100929540645a797d97 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:44.918481 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffc148-f44e-4947-bfe5-93803e424d8b None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:44.919059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffc148-f44e-4947-bfe5-93803e424d8b None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:44.919059 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffc148-f44e-4947-bfe5-93803e424d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:44.961520 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d7ffc148-f44e-4947-bfe5-93803e424d8b None None] MFA Rules not processed for user `cd54bb2ffebc46f08f2b57cb60122376`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:44.962843 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d7ffc148-f44e-4947-bfe5-93803e424d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:45.055506 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1471/2934] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:45.068872 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:45.070665 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:45.142948 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5dc459-bc18-42ef-90d9-8861f936fd32 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:45.146850 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:45.147635 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:45.147986 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:45.148350 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:45.218088 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:45.219654 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:45.232358 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1464/2935] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:45.341400 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:45.341688 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:45.342035 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:45.362433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:45.363692 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:45.370066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:45.428441 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1472/2936] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:45.477160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:45.478841 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:45.514216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:45.555428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217c27c-d98f-4c0c-8002-6c1a6524986c service glance] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:45.561500 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfff29-5756-406e-bd1a-8834b17d8cc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:45.562090 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfff29-5756-406e-bd1a-8834b17d8cc2 service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:45.562377 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfff29-5756-406e-bd1a-8834b17d8cc2 service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:45.562651 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfff29-5756-406e-bd1a-8834b17d8cc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:45.571589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfff29-5756-406e-bd1a-8834b17d8cc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:45.573189 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfff29-5756-406e-bd1a-8834b17d8cc2 service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:45.581556 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1465/2937] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:29:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:29:49.167767 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:49.170526 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:49.261430 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489a56d-b602-4a9e-9ada-21a30e51cba8 None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:49.269941 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:49.270943 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:49.271236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:49.271543 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] PATH_INFO: `/v3/users/a84f39ff866e48038c9d36188824c9e5` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:49.272794 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84f39ff866e48038c9d36188824c9e5)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:49.274881 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:49.338425 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84f39ff866e48038c9d36188824c9e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:49.353969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84f39ff866e48038c9d36188824c9e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:49.366239 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84f39ff866e48038c9d36188824c9e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:49.376611 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84f39ff866e48038c9d36188824c9e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:49.377323 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84f39ff866e48038c9d36188824c9e5'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:49.383945 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1473/2938] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:49 2022] DELETE /identity/v3/users/a84f39ff866e48038c9d36188824c9e5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:49.388550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:49.389142 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:49.389630 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:49.409871 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:49.411462 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:49.507214 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1466/2939] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:49.516046 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:49.517696 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:49.561457 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:49.592322 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e84695-4427-4e3a-9db3-d96e99a3b8fd admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:49.597793 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17621b7-9b5c-473f-a61d-1882c7a4f3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:49.598640 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17621b7-9b5c-473f-a61d-1882c7a4f3de service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:49.598928 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17621b7-9b5c-473f-a61d-1882c7a4f3de service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:49.599210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17621b7-9b5c-473f-a61d-1882c7a4f3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:49.614873 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17621b7-9b5c-473f-a61d-1882c7a4f3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:49.616404 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17621b7-9b5c-473f-a61d-1882c7a4f3de service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:49.631969 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1474/2940] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:50.012395 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:50.014071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:50.032991 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d13c4-2eee-4347-9ca0-d833a51ee06f None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:50.042904 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:50.044087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:50.044452 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:50.046287 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] PATH_INFO: `/v3/projects/75b686945ee4451daae3d981f261061f` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:50.051109 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b686945ee4451daae3d981f261061f)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:50.054178 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:50.156463 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b686945ee4451daae3d981f261061f', 'request_id': 'req-046dc066-17e5-431f-9f26-f3a425a2d4f3'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:50.168794 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b686945ee4451daae3d981f261061f', 'request_id': 'req-046dc066-17e5-431f-9f26-f3a425a2d4f3'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:50.191354 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b686945ee4451daae3d981f261061f', 'request_id': 'req-046dc066-17e5-431f-9f26-f3a425a2d4f3'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:50.212543 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] The token cache is being invalidate because project 75b686945ee4451daae3d981f261061f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:50.212969 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:50.214107 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:50.216335 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1467/2941] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:50 2022] DELETE /identity/v3/projects/75b686945ee4451daae3d981f261061f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:54.692811 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:54.693321 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:54.693429 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:54.711554 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] MFA Rules not processed for user `7e360d7c3b634c9eb26f1068b22ea56b`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:54.712762 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:54.771802 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1475/2942] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:29:54.809087 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:54.811004 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:54.857440 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:54.874900 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc066-17e5-431f-9f26-f3a425a2d4f3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:54.878750 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ca341-5311-4992-bac5-03fbec0c5f72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f733becb24b14cd5842de72b62d051b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:54.879437 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ca341-5311-4992-bac5-03fbec0c5f72 service glance] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:54.879716 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ca341-5311-4992-bac5-03fbec0c5f72 service glance] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:54.879988 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ca341-5311-4992-bac5-03fbec0c5f72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:54.881877 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d28ca341-5311-4992-bac5-03fbec0c5f72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:54.955734 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ca341-5311-4992-bac5-03fbec0c5f72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:54.957196 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ca341-5311-4992-bac5-03fbec0c5f72 service glance] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:54.969780 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1468/2943] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:55.472635 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:55.474330 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:55.556373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f9e2-b5eb-4b48-8fb7-9cd6600ec3da None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:55.560882 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:55.561783 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:55.562066 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:55.562343 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] PATH_INFO: `/v3/users/312fba0e9b894db290fb2d48d0d704e9` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:55.563756 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=312fba0e9b894db290fb2d48d0d704e9)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:55.565727 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:55.615021 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312fba0e9b894db290fb2d48d0d704e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:55.626776 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312fba0e9b894db290fb2d48d0d704e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:55.634587 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312fba0e9b894db290fb2d48d0d704e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:55.641806 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312fba0e9b894db290fb2d48d0d704e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:55.642449 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312fba0e9b894db290fb2d48d0d704e9'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:55.644564 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:29:55.649403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1476/2944] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:29:55 2022] DELETE /identity/v3/users/312fba0e9b894db290fb2d48d0d704e9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:29:55.654203 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:55.654637 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:55.654914 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:55.672480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] MFA Rules not processed for user `546aa1cc97e64f3e87a1b21e2088baa8`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:29:55.673746 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:55.737379 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1469/2945] 10.222.0.87 () {58 vars in 957 bytes} [Fri Jun 17 13:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 13:29:55.747298 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:55.749043 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:55.812347 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d6bee-9bbc-48cf-9806-f261dba8e117 admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:55.816216 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b933-d9fd-4a7b-a8ff-3c4b3f6dd5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:55.816859 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b933-d9fd-4a7b-a8ff-3c4b3f6dd5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:55.817131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b933-d9fd-4a7b-a8ff-3c4b3f6dd5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:55.817418 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b933-d9fd-4a7b-a8ff-3c4b3f6dd5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:55.826681 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1b933-d9fd-4a7b-a8ff-3c4b3f6dd5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:55.828161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1b933-d9fd-4a7b-a8ff-3c4b3f6dd5a8 service neutron] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:55.837788 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1477/2946] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:29:56.127076 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:29:56.128617 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:29:56.141910 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d529d1-e40b-44bb-9027-de6506d267d0 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:29:56.146968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:29:56.147958 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:29:56.148191 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:29:56.148468 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] PATH_INFO: `/v3/projects/9f83838c08d142038c759ee8d51a9262` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:29:56.149714 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f83838c08d142038c759ee8d51a9262)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:29:56.151677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:29:56.200957 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f83838c08d142038c759ee8d51a9262', 'request_id': 'req-d4633457-fedd-4294-a529-80b31af58f60'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:56.207589 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f83838c08d142038c759ee8d51a9262', 'request_id': 'req-d4633457-fedd-4294-a529-80b31af58f60'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:56.213546 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f83838c08d142038c759ee8d51a9262', 'request_id': 'req-d4633457-fedd-4294-a529-80b31af58f60'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:56.226959 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] The token cache is being invalidate because project 9f83838c08d142038c759ee8d51a9262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97010) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:29:56.227241 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:56.227915 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:29:56.229467 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1470/2947] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:29:56 2022] DELETE /identity/v3/projects/9f83838c08d142038c759ee8d51a9262 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:30:00.759591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] REQUEST_METHOD: `POST` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:30:00.760068 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:30:00.760293 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:30:00.778223 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.auth.core [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97009) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:30:00.779591 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:00.838710 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1478/2948] 10.222.0.87 () {58 vars in 1009 bytes} [Fri Jun 17 13:30:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:30:00.850771 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:30:00.852378 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:00.860006 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:30:00.930017 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633457-fedd-4294-a529-80b31af58f60 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:30:00.934806 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:30:00.935497 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:30:00.935833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:30:00.936705 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:30:00.945381 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:30:00.947155 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:30:00.956459 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1471/2949] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:30:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:30:10.629695 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:30:10.631207 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:10.754729 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e5458-e8e2-41f7-826b-93bd2dbe978a None None] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:30:10.760375 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:30:10.761053 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:30:10.761338 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:30:10.761622 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:30:10.763606 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:10.778381 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:30:10.780345 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:30:10.792458 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1479/2950] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:30:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:30:10.862353 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:30:10.864386 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:10.959808 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c0adc-1533-47ee-911a-9e4b2d47abce service swift] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:30:10.965833 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:30:10.966727 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:30:10.967292 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:30:10.967751 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:30:10.970382 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:10.986184 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:30:10.988132 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:30:11.000002 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1472/2951] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:30:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:30:15.259655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:30:15.261655 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:15.279734 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1aafb-cdd1-488e-b531-a2257ce4adce service placement] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:30:15.286631 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:30:15.287930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:30:15.287930 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:30:15.288116 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:30:15.290236 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:15.305373 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:30:15.307260 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:30:15.320465 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1480/2952] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:30:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:30:20.995034 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=64 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:30:25.317016 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=94 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:30:26.763946 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:30:26.765661 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:26.826228 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55aa8-c7a0-4246-8631-6b0c335ed97e service nova] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:30:26.830454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8727bc-eb2f-4643-8f03-6d43ad7c66cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40f6ae0e5e164461b4c51b829be9a7d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:30:26.831122 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8727bc-eb2f-4643-8f03-6d43ad7c66cd service neutron] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:30:26.831434 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8727bc-eb2f-4643-8f03-6d43ad7c66cd service neutron] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:30:26.831711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8727bc-eb2f-4643-8f03-6d43ad7c66cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:30:26.834314 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-7c8727bc-eb2f-4643-8f03-6d43ad7c66cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:26.899359 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8727bc-eb2f-4643-8f03-6d43ad7c66cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:30:26.900842 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8727bc-eb2f-4643-8f03-6d43ad7c66cd service neutron] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:30:26.910484 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1473/2953] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:30:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:30:28.811737 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:30:28.825100 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004e370-ec27-4359-9e3a-c86f50eca65f service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:30:28.830816 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abaa59b4e88642c681e9563295f6fad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:30:28.831527 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:30:28.831809 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:30:28.832697 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:30:28.835348 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:30:28.849170 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:30:28.851160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:30:28.863050 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1481/2954] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:30:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 13:30:36.906160 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=50 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:30:38.858551 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=6 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:31:46.006831 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] REQUEST_METHOD: `POST` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:31:46.007428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:31:46.007428 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:31:46.026861 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.auth.core [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] MFA Rules not processed for user `cac0c2cb80a34d608aba65ba0c628f24`. Rule list: `[]` (Enabled: `True`). {{(pid=97010) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 13:31:46.028171 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:31:46.036550 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1474/2955] 10.222.0.87 () {58 vars in 967 bytes} [Fri Jun 17 13:31:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 13:31:46.080779 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:31:46.092970 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294a11c-2e0d-4bfa-bbad-c78215708edb service nova] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:31:46.098210 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a31e162778e4824b7bafd5dcd9b4b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:31:46.098869 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] REQUEST_METHOD: `GET` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:31:46.099131 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:31:46.099433 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:31:46.109403 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:31:46.111029 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:31:46.120977 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1482/2956] 10.222.0.87 () {60 vars in 1384 bytes} [Fri Jun 17 13:31:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 13:31:53.076480 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:31:53.077972 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:31:53.149977 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0798b21-3189-4807-86b2-7416af205d30 None None] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:31:53.154743 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:31:53.155677 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:31:53.155966 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:31:53.156254 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] PATH_INFO: `/v3/users/7e360d7c3b634c9eb26f1068b22ea56b` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:31:53.157478 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e360d7c3b634c9eb26f1068b22ea56b)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:31:53.159535 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:31:53.214161 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e360d7c3b634c9eb26f1068b22ea56b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.225994 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e360d7c3b634c9eb26f1068b22ea56b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.234056 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e360d7c3b634c9eb26f1068b22ea56b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.241306 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e360d7c3b634c9eb26f1068b22ea56b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.242049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e360d7c3b634c9eb26f1068b22ea56b'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.248713 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1475/2957] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:31:53 2022] DELETE /identity/v3/users/7e360d7c3b634c9eb26f1068b22ea56b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:31:53.429925 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:31:53.442436 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32eb0f-5936-49f6-85f4-f8203ff629ad service swift] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:31:53.506000 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:31:53.506874 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:31:53.507160 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:31:53.507463 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] PATH_INFO: `/v3/projects/362880716d884b098bab99cfcb5ed2aa` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:31:53.508674 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=362880716d884b098bab99cfcb5ed2aa)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:31:53.510694 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:31:53.560972 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362880716d884b098bab99cfcb5ed2aa', 'request_id': 'req-f3702e6f-cff1-48fa-9278-72d507b36c3e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.567161 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362880716d884b098bab99cfcb5ed2aa', 'request_id': 'req-f3702e6f-cff1-48fa-9278-72d507b36c3e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.573408 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362880716d884b098bab99cfcb5ed2aa', 'request_id': 'req-f3702e6f-cff1-48fa-9278-72d507b36c3e'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.587166 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] The token cache is being invalidate because project 362880716d884b098bab99cfcb5ed2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:31:53.587570 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.588509 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.590385 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1483/2958] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:31:53 2022] DELETE /identity/v3/projects/362880716d884b098bab99cfcb5ed2aa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:31:53.594223 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:31:53.595711 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:31:53.609454 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93f1b3-7689-4c27-b9a3-ec88dd4463b3 admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:31:53.669064 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:31:53.669968 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] REQUEST_METHOD: `DELETE` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:31:53.670259 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:31:53.670536 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] PATH_INFO: `/v3/users/cd54bb2ffebc46f08f2b57cb60122376` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:31:53.671936 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd54bb2ffebc46f08f2b57cb60122376)` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:31:53.673962 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:31:53.688216 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:31:53.726188 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd54bb2ffebc46f08f2b57cb60122376'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.739049 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd54bb2ffebc46f08f2b57cb60122376'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.746837 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd54bb2ffebc46f08f2b57cb60122376'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.754812 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd54bb2ffebc46f08f2b57cb60122376'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.755612 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.notifications [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd54bb2ffebc46f08f2b57cb60122376'} {{(pid=97010) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:53.762089 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1476/2959] 10.222.0.87 () {58 vars in 1223 bytes} [Fri Jun 17 13:31:53 2022] DELETE /identity/v3/users/cd54bb2ffebc46f08f2b57cb60122376 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:31:53.946309 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Authenticating user token {{(pid=97009) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:31:53.949030 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.fernet_utils [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97009) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:31:53.962230 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3702e6f-cff1-48fa-9278-72d507b36c3e admin admin] Validating token access rules against request {{(pid=97009) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:31:54.021470 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546aa1cc97e64f3e87a1b21e2088baa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cba89e292b47ef9da2168f3c55b232', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97009) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:31:54.022366 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] REQUEST_METHOD: `DELETE` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:31:54.022643 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] SCRIPT_NAME: `/identity` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:31:54.022913 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] PATH_INFO: `/v3/projects/f390ded3fbe64c83b577ebcf11af278c` {{(pid=97009) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:31:54.024147 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f390ded3fbe64c83b577ebcf11af278c)` {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:31:54.026612 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] RBAC: Authorization granted {{(pid=97009) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:31:54.051178 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:31:54.078775 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f390ded3fbe64c83b577ebcf11af278c', 'request_id': 'req-d366e586-6801-4995-b17f-fd56a876cfac'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:54.084937 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f390ded3fbe64c83b577ebcf11af278c', 'request_id': 'req-d366e586-6801-4995-b17f-fd56a876cfac'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:54.091282 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f390ded3fbe64c83b577ebcf11af278c', 'request_id': 'req-d366e586-6801-4995-b17f-fd56a876cfac'} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:54.105415 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] The token cache is being invalidate because project f390ded3fbe64c83b577ebcf11af278c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=97009) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 13:31:54.105708 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:54.106328 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG keystone.notifications [None req-d366e586-6801-4995-b17f-fd56a876cfac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=97009) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 13:31:54.107965 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: [pid: 97009|app: 0|req: 1484/2960] 10.222.0.87 () {58 vars in 1232 bytes} [Fri Jun 17 13:31:53 2022] DELETE /identity/v3/projects/f390ded3fbe64c83b577ebcf11af278c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 13:32:03.758130 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:32:04.102052 dv-cinder-846172-2-iscsi devstack@keystone.service[97009]: DEBUG dbcounter [-] [97009] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=97009) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 13:35:13.657789 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Authenticating user token {{(pid=97010) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 13:35:13.660220 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:35:13.822239 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabc88e-fd2f-43b0-ab38-416895cecfab admin admin] Validating token access rules against request {{(pid=97010) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 13:35:13.893488 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4eb380-fc1e-4274-8e41-abe2c7ac67e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34865acbc294b3b851bcea0cf6b442e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f70dff20b8427180b9b3b5a8b418cd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=97010) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 13:35:13.894071 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb380-fc1e-4274-8e41-abe2c7ac67e3 service placement] REQUEST_METHOD: `GET` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 13:35:13.894361 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb380-fc1e-4274-8e41-abe2c7ac67e3 service placement] SCRIPT_NAME: `/identity` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 13:35:13.894644 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb380-fc1e-4274-8e41-abe2c7ac67e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=97010) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 13:35:13.896863 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.fernet_utils [None req-ee4eb380-fc1e-4274-8e41-abe2c7ac67e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=97010) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 13:35:13.910366 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4eb380-fc1e-4274-8e41-abe2c7ac67e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 13:35:13.912058 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4eb380-fc1e-4274-8e41-abe2c7ac67e3 service placement] RBAC: Authorization granted {{(pid=97010) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 13:35:13.923857 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: [pid: 97010|app: 0|req: 1477/2961] 10.222.0.87 () {60 vars in 1365 bytes} [Fri Jun 17 13:35:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 13:35:23.918920 dv-cinder-846172-2-iscsi devstack@keystone.service[97010]: DEBUG dbcounter [-] [97010] Writing DB stats keystone:SELECT=80 {{(pid=97010) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}